[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:34 dist-test-slave-w206 krb5kdc[31011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:34 dist-test-slave-w206 krb5kdc[31011](info): set up 2 sockets
Aug 10 06:35:34 dist-test-slave-w206 krb5kdc[31011](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 06:35:35.437963 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:35 dist-test-slave-w206 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:35.564399 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:44427
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:44427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:35.669838 31024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:35.670059 31024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:35.670084 31024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:35.671340 31024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:35.671384 31024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:35.671399 31024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:35.671413 31024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:35.671425 31024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:35.673049 31024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:44427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:44427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:35.673382 31024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:35.673625 31024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:35.680450 31029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:35.681349 31032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:35.681576 31030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:35.681650 31024 server_base.cc:1034] running on GCE node
I20260810 06:35:35.681967 31024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:35.682296 31024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:35.683497 31024 hybrid_clock.cc:648] HybridClock initialized: now 1786343735683448 us; error 65 us; skew 500 ppm
Aug 10 06:35:35 dist-test-slave-w206 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343735, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:35.686282 31024 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:35.686916 31024 webserver.cc:460] Webserver started at http://127.30.71.62:42389/ using document root <none> and password file <none>
I20260810 06:35:35.687254 31024 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:35.687333 31024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:35.687521 31024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:35.688968 31024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "8011999e14454539a2c0b6c3e31dc0e9"
format_stamp: "Formatted at 2026-08-10 06:35:35 on dist-test-slave-w206"
server_key: "c8dcc8a1156791b4d4f7651856e4bc38"
server_key_iv: "b1c7865d4441aab7594f4ae1d233298a"
server_key_version: "encryptionkey@0"
I20260810 06:35:35.689708 31024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "8011999e14454539a2c0b6c3e31dc0e9"
format_stamp: "Formatted at 2026-08-10 06:35:35 on dist-test-slave-w206"
server_key: "c8dcc8a1156791b4d4f7651856e4bc38"
server_key_iv: "b1c7865d4441aab7594f4ae1d233298a"
server_key_version: "encryptionkey@0"
I20260810 06:35:35.691771 31024 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 06:35:35.692893 31040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:35.693266 31024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:35.693410 31024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "8011999e14454539a2c0b6c3e31dc0e9"
format_stamp: "Formatted at 2026-08-10 06:35:35 on dist-test-slave-w206"
server_key: "c8dcc8a1156791b4d4f7651856e4bc38"
server_key_iv: "b1c7865d4441aab7594f4ae1d233298a"
server_key_version: "encryptionkey@0"
I20260810 06:35:35.693509 31024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:35.716859 31024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:35.719602 31024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:35.719836 31024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:35.725289 31024 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:44427
I20260810 06:35:35.725355 31096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:44427 every 8 connection(s)
I20260810 06:35:35.725953 31024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:35.727452 31097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:35.730722 31097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9: Bootstrap starting.
I20260810 06:35:35.730690 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31024
I20260810 06:35:35.730825 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:35.731812 31004 external_mini_cluster.cc:1428] Setting key e2f6e28b3f4dbb9efedd4f327cce9612
I20260810 06:35:35.731925 31097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:35.732386 31097 log.cc:824] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:35.733592 31097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9: No bootstrap required, opened a new log
I20260810 06:35:35.736804 31097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011999e14454539a2c0b6c3e31dc0e9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 44427 } }
I20260810 06:35:35.737098 31097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:35.737211 31097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8011999e14454539a2c0b6c3e31dc0e9, State: Initialized, Role: FOLLOWER
I20260810 06:35:35.737361 31097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011999e14454539a2c0b6c3e31dc0e9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 44427 } }
I20260810 06:35:35.737493 31097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:35.737530 31097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:35.737579 31097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:35.738898 31097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011999e14454539a2c0b6c3e31dc0e9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 44427 } }
I20260810 06:35:35.739152 31097 leader_election.cc:302] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8011999e14454539a2c0b6c3e31dc0e9; no voters:
I20260810 06:35:35.739442 31097 leader_election.cc:288] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:35.739476 31102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:35.739691 31102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [term 1 LEADER]: Becoming Leader. State: Replica: 8011999e14454539a2c0b6c3e31dc0e9, State: Running, Role: LEADER
I20260810 06:35:35.739872 31102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011999e14454539a2c0b6c3e31dc0e9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 44427 } }
I20260810 06:35:35.740007 31097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:35.740514 31103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8011999e14454539a2c0b6c3e31dc0e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011999e14454539a2c0b6c3e31dc0e9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 44427 } } }
I20260810 06:35:35.740617 31103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:35.740509 31104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8011999e14454539a2c0b6c3e31dc0e9. Latest consensus state: current_term: 1 leader_uuid: "8011999e14454539a2c0b6c3e31dc0e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8011999e14454539a2c0b6c3e31dc0e9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 44427 } } }
I20260810 06:35:35.740840 31104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:35.741184 31106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:35.742574 31106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:35.748844 31106 catalog_manager.cc:1347] Generated new cluster ID: e3f11f125c9149df96f32ddecab84ed9
I20260810 06:35:35.748927 31106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:35:35 dist-test-slave-w206 krb5kdc[31011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:35.755141 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:35.733445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33528 (local address 127.30.71.62:44427)
0810 06:35:35.736054 (+ 2609us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:35.736062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:35.736098 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:35:35.736743 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:35.736749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:35.736815 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:35.736940 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:35.739694 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:35.741973 (+ 2279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:35.744407 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:35.745267 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:35.745417 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:35.750445 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:35.750450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:35.750450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:35.750462 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:35.752747 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:35.753317 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:35.753320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:35.753320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:35.753444 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:35.753772 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:35.753773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:35.753773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:35.754098 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:35.754253 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:35.754683 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 06:35:35.754835 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2502,"thread_start_us":163,"threads_started":1}
I20260810 06:35:35.770599 31106 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:35.771413 31106 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:35.776459 31106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8011999e14454539a2c0b6c3e31dc0e9: Generated new TSK 0
I20260810 06:35:35.776700 31106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:35.814591 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:44427
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:35.949955 31123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:35.950290 31123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:35.950335 31123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:35.952493 31123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:35.952574 31123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:35.952625 31123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:35.955286 31123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:44427
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:35.955622 31123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:35.956032 31123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:35.962745 31129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:35.962962 31128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:35.962762 31131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:35.963831 31123 server_base.cc:1034] running on GCE node
I20260810 06:35:35.964072 31123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:35.964394 31123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:35.965586 31123 hybrid_clock.cc:648] HybridClock initialized: now 1786343735965558 us; error 46 us; skew 500 ppm
Aug 10 06:35:35 dist-test-slave-w206 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343735, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:35.968777 31123 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:35.969259 31123 webserver.cc:460] Webserver started at http://127.30.71.1:38985/ using document root <none> and password file <none>
I20260810 06:35:35.969522 31123 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:35.969599 31123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:35.969772 31123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:35.971108 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "6bf0a6b01ade41cd9b5ee8739e67b734"
format_stamp: "Formatted at 2026-08-10 06:35:35 on dist-test-slave-w206"
server_key: "31a9439612d9a7b4135c1aca747b6ed3"
server_key_iv: "ea71ec420232549b76641b90b9473e86"
server_key_version: "encryptionkey@0"
I20260810 06:35:35.971585 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "6bf0a6b01ade41cd9b5ee8739e67b734"
format_stamp: "Formatted at 2026-08-10 06:35:35 on dist-test-slave-w206"
server_key: "31a9439612d9a7b4135c1aca747b6ed3"
server_key_iv: "ea71ec420232549b76641b90b9473e86"
server_key_version: "encryptionkey@0"
I20260810 06:35:35.973279 31123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:35:35.974162 31138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:35.974370 31123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:35.974483 31123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "6bf0a6b01ade41cd9b5ee8739e67b734"
format_stamp: "Formatted at 2026-08-10 06:35:35 on dist-test-slave-w206"
server_key: "31a9439612d9a7b4135c1aca747b6ed3"
server_key_iv: "ea71ec420232549b76641b90b9473e86"
server_key_version: "encryptionkey@0"
I20260810 06:35:35.974558 31123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:35.988660 31123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:35.990658 31123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:35.990826 31123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:35.991127 31123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:35.991467 31123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:35.991535 31123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:35.991578 31123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:35.991604 31123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:36.001539 31123 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:39803
I20260810 06:35:36.001581 31262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:39803 every 8 connection(s)
I20260810 06:35:36.001950 31123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:36.002637 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31123
I20260810 06:35:36.002741 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:36.002948 31004 external_mini_cluster.cc:1428] Setting key 1b8369bc38f38d9e397630e05e5144f9
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343735, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:35:36.013975 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.003001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:44325 (local address 127.30.71.62:44427)
0810 06:35:36.003078 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:36.003080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:36.003326 (+ 246us) server_negotiation.cc:407] Connection header received
0810 06:35:36.004058 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:36.004059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:36.004078 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:36.004166 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:36.005368 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.006030 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.007681 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.007911 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.007982 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.010685 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:36.010688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:36.010688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:36.010697 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:36.012427 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.012897 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.012898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.012899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.012965 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.013261 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.013263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.013263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.013446 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:36.013545 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:36.013808 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 06:35:36.013901 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:35:36.014052 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.002956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:44427 (local address 127.30.71.1:44325)
0810 06:35:36.003177 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:36.003229 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:36.003906 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:36.004227 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:36.004231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:36.005010 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:36.005230 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:36.005237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.006134 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.006136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:36.007579 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:36.007582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.008011 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.008012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:36.008171 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.008742 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:36.008752 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:36.010570 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:36.012544 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:36.012552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:36.012554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:36.012800 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:36.013037 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:36.013038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:36.013038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:36.013155 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:36.013530 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:36.013535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:36.013654 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 06:35:36.013757 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:35:36.013883 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
I20260810 06:35:36.015053 31264 heartbeater.cc:347] Connected to a master server at 127.30.71.62:44427
I20260810 06:35:36.015156 31264 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:36.015401 31264 heartbeater.cc:511] Master 127.30.71.62:44427 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
I20260810 06:35:36.016554 31059 ts_manager.cc:194] Registered new tserver with Master: 6bf0a6b01ade41cd9b5ee8739e67b734 (127.30.71.1:39803)
I20260810 06:35:36.017912 31059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:44325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:36.034107 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:44427
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:36.126772 31269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:36.127046 31269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:36.127090 31269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:36.128217 31269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:36.128253 31269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:36.128280 31269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:36.129758 31269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:44427
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:36.129946 31269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:36.130125 31269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:36.135272 31279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:36.135251 31276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:36.135248 31269 server_base.cc:1034] running on GCE node
W20260810 06:35:36.135326 31275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:36.135687 31269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:36.135968 31269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:36.137135 31269 hybrid_clock.cc:648] HybridClock initialized: now 1786343736137123 us; error 44 us; skew 500 ppm
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:36.140084 31269 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:36.140611 31269 webserver.cc:460] Webserver started at http://127.30.71.2:37607/ using document root <none> and password file <none>
I20260810 06:35:36.140863 31269 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:36.140933 31269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:36.141072 31269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:36.142247 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "d25db3c0ae0e4131bd8b7b247b8341da"
format_stamp: "Formatted at 2026-08-10 06:35:36 on dist-test-slave-w206"
server_key: "28829e96de936eb57d0964789ccb75a0"
server_key_iv: "d9b55c45336e8e6f34da631a816d39e5"
server_key_version: "encryptionkey@0"
I20260810 06:35:36.142657 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "d25db3c0ae0e4131bd8b7b247b8341da"
format_stamp: "Formatted at 2026-08-10 06:35:36 on dist-test-slave-w206"
server_key: "28829e96de936eb57d0964789ccb75a0"
server_key_iv: "d9b55c45336e8e6f34da631a816d39e5"
server_key_version: "encryptionkey@0"
I20260810 06:35:36.144258 31269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:36.145135 31286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:36.145341 31269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:36.145443 31269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "d25db3c0ae0e4131bd8b7b247b8341da"
format_stamp: "Formatted at 2026-08-10 06:35:36 on dist-test-slave-w206"
server_key: "28829e96de936eb57d0964789ccb75a0"
server_key_iv: "d9b55c45336e8e6f34da631a816d39e5"
server_key_version: "encryptionkey@0"
I20260810 06:35:36.145525 31269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:36.179204 31269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:36.181844 31269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:36.182078 31269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:36.182348 31269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:36.182680 31269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:36.182736 31269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:36.182772 31269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:36.182821 31269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:36.192474 31269 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:42323
I20260810 06:35:36.192523 31414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:42323 every 8 connection(s)
I20260810 06:35:36.192909 31269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:36.200747 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31269
I20260810 06:35:36.200857 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:36.201071 31004 external_mini_cluster.cc:1428] Setting key 02a8b4bcf4b9449f57234e52b6e15f8a
I20260810 06:35:36.204365 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.193811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:39473 (local address 127.30.71.62:44427)
0810 06:35:36.193880 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:36.193883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:36.194117 (+ 234us) server_negotiation.cc:407] Connection header received
0810 06:35:36.194675 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:36.194676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:36.194689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:36.194735 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:36.195779 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.196301 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.197485 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.197774 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.197850 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.200583 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:36.200585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:36.200585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:36.200593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:36.202349 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.203064 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.203065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.203066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.203122 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.203475 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.203476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.203476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.203588 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:36.203643 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:36.204096 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 06:35:36.204223 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:35:36.204550 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.193786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:44427 (local address 127.30.71.2:39473)
0810 06:35:36.193985 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:36.194024 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:36.194552 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:36.194804 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:36.194808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:36.195437 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:36.195675 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:36.195680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.196406 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.196407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:36.197389 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:36.197391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.197903 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.197904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:36.198082 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.198821 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:36.198831 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:36.200452 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:36.202440 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:36.202448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:36.202450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:36.202904 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:36.203201 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:36.203202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:36.203202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:36.203395 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:36.203668 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:36.203675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:36.203900 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 06:35:36.204036 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:35:36.204251 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260810 06:35:36.205190 31415 heartbeater.cc:347] Connected to a master server at 127.30.71.62:44427
I20260810 06:35:36.205322 31415 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:36.205520 31415 heartbeater.cc:511] Master 127.30.71.62:44427 requested a full tablet report, sending...
I20260810 06:35:36.205932 31059 ts_manager.cc:194] Registered new tserver with Master: d25db3c0ae0e4131bd8b7b247b8341da (127.30.71.2:42323)
I20260810 06:35:36.206518 31059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:39473
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:36.234337 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:44427
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:36.328346 31422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:36.328552 31422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:36.328577 31422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:36.329856 31422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:36.329908 31422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:36.329937 31422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:36.331996 31422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:44427
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:36.332297 31422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:36.332590 31422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:36.338572 31427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:36.338577 31431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:36.338577 31429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:36.338850 31422 server_base.cc:1034] running on GCE node
I20260810 06:35:36.339129 31422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:36.339414 31422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:36.340564 31422 hybrid_clock.cc:648] HybridClock initialized: now 1786343736340551 us; error 35 us; skew 500 ppm
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:36.343482 31422 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:36.343969 31422 webserver.cc:460] Webserver started at http://127.30.71.3:46209/ using document root <none> and password file <none>
I20260810 06:35:36.344226 31422 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:36.344295 31422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:36.344445 31422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:36.345657 31422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "10125ef35e9745ed9ca1ba917553815a"
format_stamp: "Formatted at 2026-08-10 06:35:36 on dist-test-slave-w206"
server_key: "b3732fa882ab82cc945dbcf569470229"
server_key_iv: "227d1c0398225b75ec01decca372e037"
server_key_version: "encryptionkey@0"
I20260810 06:35:36.346112 31422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "10125ef35e9745ed9ca1ba917553815a"
format_stamp: "Formatted at 2026-08-10 06:35:36 on dist-test-slave-w206"
server_key: "b3732fa882ab82cc945dbcf569470229"
server_key_iv: "227d1c0398225b75ec01decca372e037"
server_key_version: "encryptionkey@0"
I20260810 06:35:36.347781 31422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:36.348789 31442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:36.349057 31422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:36.349164 31422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "10125ef35e9745ed9ca1ba917553815a"
format_stamp: "Formatted at 2026-08-10 06:35:36 on dist-test-slave-w206"
server_key: "b3732fa882ab82cc945dbcf569470229"
server_key_iv: "227d1c0398225b75ec01decca372e037"
server_key_version: "encryptionkey@0"
I20260810 06:35:36.349277 31422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:36.360467 31422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:36.362919 31422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:36.363129 31422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:36.363389 31422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:36.363703 31422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:36.363758 31422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:36.363806 31422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:36.363837 31422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:36.374322 31422 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:38091
I20260810 06:35:36.374365 31564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:38091 every 8 connection(s)
I20260810 06:35:36.374734 31422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:36.382252 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31422
I20260810 06:35:36.382340 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:36.382512 31004 external_mini_cluster.cc:1428] Setting key 99590582a881a8e6be7796df436d2803
I20260810 06:35:36.385366 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.375714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:36443 (local address 127.30.71.62:44427)
0810 06:35:36.375798 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:36.375800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:36.375970 (+ 170us) server_negotiation.cc:407] Connection header received
0810 06:35:36.376512 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:36.376513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:36.376523 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:36.376578 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:36.377672 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.378275 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.379342 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.379564 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.379645 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.381926 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:36.381928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:36.381928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:36.381936 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:36.384076 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.384488 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.384489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.384489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.384552 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.384780 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.384781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.384781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.384896 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:36.384959 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:36.385223 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 06:35:36.385296 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:35:36.385504 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.375657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:44427 (local address 127.30.71.3:36443)
0810 06:35:36.375839 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:36.375877 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:36.376424 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:36.376633 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:36.376637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:36.377317 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:36.377577 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:36.377583 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.378357 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.378358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:36.379253 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:36.379256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.379653 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.379653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:36.379762 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.380324 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:36.380333 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:36.381835 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:36.384171 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:36.384177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:36.384179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:36.384416 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:36.384613 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:36.384614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:36.384614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:36.384715 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:36.384956 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:36.384960 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:36.385074 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 06:35:36.385155 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:35:36.385314 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":116,"threads_started":1}
I20260810 06:35:36.385946 31565 heartbeater.cc:347] Connected to a master server at 127.30.71.62:44427
I20260810 06:35:36.386070 31565 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:36.386281 31565 heartbeater.cc:511] Master 127.30.71.62:44427 requested a full tablet report, sending...
I20260810 06:35:36.386675 31059 ts_manager.cc:194] Registered new tserver with Master: 10125ef35e9745ed9ca1ba917553815a (127.30.71.3:38091)
I20260810 06:35:36.387077 31059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:36443
I20260810 06:35:36.395133 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:35:36.421830 31572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.413615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.30.71.1:39803)
0810 06:35:36.413858 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:36.413863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:36.413875 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:35:36.413914 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:36.413916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:36.413933 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:36.414068 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:36.414618 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.415678 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.416386 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.416728 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.416795 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.419132 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:36.419140 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:36.419141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:36.419148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:36.420739 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.421052 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.421055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.421055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.421129 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.421288 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.421289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.421289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.421423 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:36.421537 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:36.421598 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 06:35:36.421662 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":125,"threads_started":1}
W20260810 06:35:36.422307 31168 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37026
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:35:36.447889 31572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:36.440024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.30.71.1:39803)
0810 06:35:36.440108 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:36.440110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:36.440121 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:35:36.440183 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:36.440184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:36.440193 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:36.440244 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:36.440961 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.441750 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.442487 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:36.442718 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:36.442789 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:36.445199 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:36.445200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:36.445201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:36.445211 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:36.446646 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.447109 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.447110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.447110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.447168 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:36.447395 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:36.447395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:36.447396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:36.447491 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:36.447585 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:36.447670 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 06:35:36.447749 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:35:36.448725 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31123
I20260810 06:35:36.456073 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31269
I20260810 06:35:36.462456 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31422
I20260810 06:35:36.467936 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31024
2026-08-10T06:35:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2170 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31580](info): set up 2 sockets
Aug 10 06:35:36 dist-test-slave-w206 krb5kdc[31580](info): commencing operation
krb5kdc: starting...
W20260810 06:35:37.652599 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:37 dist-test-slave-w206 krb5kdc[31580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:37.778261 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36351
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:36351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:37.862782 31593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:37.863051 31593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:37.863093 31593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:37.864488 31593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:37.864565 31593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:37.864591 31593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:37.864624 31593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:37.864651 31593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:37.866181 31593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:36351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:37.866487 31593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:37.866739 31593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:37.871914 31602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:37.871902 31599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:37.871960 31593 server_base.cc:1034] running on GCE node
W20260810 06:35:37.871930 31600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:37.872440 31593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:37.872718 31593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:37.873898 31593 hybrid_clock.cc:648] HybridClock initialized: now 1786343737873870 us; error 45 us; skew 500 ppm
Aug 10 06:35:37 dist-test-slave-w206 krb5kdc[31580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343737, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:37.876782 31593 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:37.877269 31593 webserver.cc:460] Webserver started at http://127.30.71.62:36039/ using document root <none> and password file <none>
I20260810 06:35:37.877525 31593 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:37.877595 31593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:37.877756 31593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:37.878876 31593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "f45ea55699cc4320bc46224d2427a32a"
format_stamp: "Formatted at 2026-08-10 06:35:37 on dist-test-slave-w206"
server_key: "eb79ccf45fc1703746300720565b2004"
server_key_iv: "4d6fb9fea6b7d3ed081f33d23c8592e2"
server_key_version: "encryptionkey@0"
I20260810 06:35:37.879360 31593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "f45ea55699cc4320bc46224d2427a32a"
format_stamp: "Formatted at 2026-08-10 06:35:37 on dist-test-slave-w206"
server_key: "eb79ccf45fc1703746300720565b2004"
server_key_iv: "4d6fb9fea6b7d3ed081f33d23c8592e2"
server_key_version: "encryptionkey@0"
I20260810 06:35:37.880941 31593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:35:37.881909 31610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:37.882143 31593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:37.882251 31593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "f45ea55699cc4320bc46224d2427a32a"
format_stamp: "Formatted at 2026-08-10 06:35:37 on dist-test-slave-w206"
server_key: "eb79ccf45fc1703746300720565b2004"
server_key_iv: "4d6fb9fea6b7d3ed081f33d23c8592e2"
server_key_version: "encryptionkey@0"
I20260810 06:35:37.882352 31593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:37.893599 31593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:37.896097 31593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:37.896273 31593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:37.900281 31593 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:36351
I20260810 06:35:37.900396 31665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:36351 every 8 connection(s)
I20260810 06:35:37.900727 31593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:37.901711 31667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:37.903709 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31593
I20260810 06:35:37.903831 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:37.904052 31004 external_mini_cluster.cc:1428] Setting key c153e6de75eb5a1d6c1a2d0a7c710a2e
I20260810 06:35:37.904307 31667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a: Bootstrap starting.
I20260810 06:35:37.905086 31667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:37.905411 31667 log.cc:824] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:37.906132 31667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a: No bootstrap required, opened a new log
I20260810 06:35:37.908308 31667 raft_consensus.cc:348] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ea55699cc4320bc46224d2427a32a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36351 } }
I20260810 06:35:37.908469 31667 raft_consensus.cc:374] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:37.908520 31667 raft_consensus.cc:729] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f45ea55699cc4320bc46224d2427a32a, State: Initialized, Role: FOLLOWER
I20260810 06:35:37.908661 31667 consensus_queue.cc:260] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ea55699cc4320bc46224d2427a32a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36351 } }
I20260810 06:35:37.908725 31667 raft_consensus.cc:388] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:37.908777 31667 raft_consensus.cc:482] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:37.908826 31667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:37.909415 31667 raft_consensus.cc:504] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ea55699cc4320bc46224d2427a32a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36351 } }
I20260810 06:35:37.909533 31667 leader_election.cc:302] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f45ea55699cc4320bc46224d2427a32a; no voters:
I20260810 06:35:37.909718 31667 leader_election.cc:288] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:37.909780 31672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:37.910009 31672 raft_consensus.cc:686] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [term 1 LEADER]: Becoming Leader. State: Replica: f45ea55699cc4320bc46224d2427a32a, State: Running, Role: LEADER
I20260810 06:35:37.910102 31667 sys_catalog.cc:564] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:37.910255 31672 consensus_queue.cc:237] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ea55699cc4320bc46224d2427a32a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36351 } }
I20260810 06:35:37.910684 31674 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f45ea55699cc4320bc46224d2427a32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ea55699cc4320bc46224d2427a32a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36351 } } }
I20260810 06:35:37.910790 31674 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:37.910938 31674 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f45ea55699cc4320bc46224d2427a32a. Latest consensus state: current_term: 1 leader_uuid: "f45ea55699cc4320bc46224d2427a32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ea55699cc4320bc46224d2427a32a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36351 } } }
I20260810 06:35:37.911046 31674 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:37.911576 31682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 06:35:37 dist-test-slave-w206 krb5kdc[31580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:37.912339 31682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:37.914234 31682 catalog_manager.cc:1347] Generated new cluster ID: 72abdf45778845b1a2e4d80b03d9d7d7
I20260810 06:35:37.914294 31682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:37.916224 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:37.904688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.30.71.62:36351)
0810 06:35:37.904901 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:37.904911 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:37.904947 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:35:37.905535 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:37.905545 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:37.905626 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:37.905794 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:37.906538 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:37.907607 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:37.908496 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:37.909129 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:37.909272 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:37.912530 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:37.912534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:37.912534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:37.912545 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:37.914601 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:37.915023 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:37.915026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:37.915026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:37.915123 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:37.915360 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:37.915361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:37.915361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:37.915608 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:37.915774 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:37.915895 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:35:37.916042 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260810 06:35:37.932643 31682 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:37.933463 31682 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:37.945042 31682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f45ea55699cc4320bc46224d2427a32a: Generated new TSK 0
I20260810 06:35:37.945269 31682 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:37.982235 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:36351
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:38.069883 31696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:38.070152 31696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:38.070183 31696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:38.071446 31696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:38.071486 31696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:38.071514 31696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:38.072976 31696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:36351
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:38.073164 31696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:38.073344 31696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:38.078595 31705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:38.078595 31702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:38.078619 31703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:38.078917 31696 server_base.cc:1034] running on GCE node
I20260810 06:35:38.079157 31696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:38.079363 31696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:38.080505 31696 hybrid_clock.cc:648] HybridClock initialized: now 1786343738080493 us; error 32 us; skew 500 ppm
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:38.083252 31696 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:38.083770 31696 webserver.cc:460] Webserver started at http://127.30.71.1:37557/ using document root <none> and password file <none>
I20260810 06:35:38.084007 31696 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:38.084053 31696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:38.084201 31696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:38.085371 31696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "c2d43df9a0084f7f9d3c160987f14339"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "d8335c6c772fa01ddfae4dd64abfbd86"
server_key_iv: "dfea0de8444221cf54b9bf86a41a0b31"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.085750 31696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "c2d43df9a0084f7f9d3c160987f14339"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "d8335c6c772fa01ddfae4dd64abfbd86"
server_key_iv: "dfea0de8444221cf54b9bf86a41a0b31"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.087277 31696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:35:38.088086 31713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.088265 31696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:35:38.088374 31696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "c2d43df9a0084f7f9d3c160987f14339"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "d8335c6c772fa01ddfae4dd64abfbd86"
server_key_iv: "dfea0de8444221cf54b9bf86a41a0b31"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.088449 31696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:38.106534 31696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:38.109230 31696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:38.109424 31696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:38.109710 31696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:38.110178 31696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:38.110239 31696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.110285 31696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:38.110314 31696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.117465 31696 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38793
I20260810 06:35:38.117612 31840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38793 every 8 connection(s)
I20260810 06:35:38.117949 31696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:38.118642 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31696
I20260810 06:35:38.118726 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:38.118911 31004 external_mini_cluster.cc:1428] Setting key f21976465d058a37f58467fc609597ac
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:35:38.129928 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.119431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:53997 (local address 127.30.71.62:36351)
0810 06:35:38.119475 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.119477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.119812 (+ 335us) server_negotiation.cc:407] Connection header received
0810 06:35:38.120933 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.120934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.120948 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.121024 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:38.121673 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.122490 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.123715 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.123965 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.124065 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.126666 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:38.126668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:38.126668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:38.126677 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:38.128425 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.128878 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.128880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.128880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.128972 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.129236 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.129238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.129238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.129359 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:38.129438 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.129726 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.129830 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:35:38.130120 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.119447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:36351 (local address 127.30.71.1:53997)
0810 06:35:38.119641 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.119714 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.120796 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.121083 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.121086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.121368 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:38.121573 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.121578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.122620 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.122622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.123551 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.123554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.124081 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.124082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.124236 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.124882 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:38.124891 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:38.126557 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:38.128525 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:38.128533 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:38.128535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:38.128783 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:38.129048 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:38.129048 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:38.129049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:38.129166 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:38.129428 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:38.129434 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:38.129587 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.129707 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.129873 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260810 06:35:38.130681 31841 heartbeater.cc:347] Connected to a master server at 127.30.71.62:36351
I20260810 06:35:38.130805 31841 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
I20260810 06:35:38.131045 31841 heartbeater.cc:511] Master 127.30.71.62:36351 requested a full tablet report, sending...
I20260810 06:35:38.131596 31629 ts_manager.cc:194] Registered new tserver with Master: c2d43df9a0084f7f9d3c160987f14339 (127.30.71.1:38793)
I20260810 06:35:38.132251 31629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:53997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:38.150512 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:36351
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:38.238482 31847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:38.238799 31847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:38.238843 31847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:38.240541 31847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:38.240619 31847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:38.240676 31847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:38.242589 31847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:36351
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:38.242914 31847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:38.243211 31847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:38.248689 31852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:38.248703 31853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:38.248740 31855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:38.248863 31847 server_base.cc:1034] running on GCE node
I20260810 06:35:38.249155 31847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:38.249434 31847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:38.250643 31847 hybrid_clock.cc:648] HybridClock initialized: now 1786343738250621 us; error 41 us; skew 500 ppm
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:38.253718 31847 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:38.254312 31847 webserver.cc:460] Webserver started at http://127.30.71.2:41257/ using document root <none> and password file <none>
I20260810 06:35:38.254585 31847 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:38.254663 31847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:38.254827 31847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:38.256044 31847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "66fd892d7a184c2090931de9d9c12b33"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "f8199504b9c8d4c07ab6823508b56add"
server_key_iv: "e69e5f690443ec2f65c04644ab992e58"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.256448 31847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "66fd892d7a184c2090931de9d9c12b33"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "f8199504b9c8d4c07ab6823508b56add"
server_key_iv: "e69e5f690443ec2f65c04644ab992e58"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.257937 31847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:35:38.258749 31864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.259084 31847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:38.259195 31847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "66fd892d7a184c2090931de9d9c12b33"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "f8199504b9c8d4c07ab6823508b56add"
server_key_iv: "e69e5f690443ec2f65c04644ab992e58"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.259289 31847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:38.279934 31847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:38.282413 31847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:38.282619 31847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:38.282907 31847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:38.283330 31847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:38.283393 31847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.283439 31847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:38.283468 31847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.290019 31847 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:46271
I20260810 06:35:38.290066 31991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:46271 every 8 connection(s)
I20260810 06:35:38.290452 31847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:35:38.297024 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31847
I20260810 06:35:38.297168 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:38.297430 31004 external_mini_cluster.cc:1428] Setting key d233bf2e93e2feea509ca81f229f40f7
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:38.301849 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.291541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:48765 (local address 127.30.71.62:36351)
0810 06:35:38.291633 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.291636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.291840 (+ 204us) server_negotiation.cc:407] Connection header received
0810 06:35:38.292932 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.292933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.292947 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.293008 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:38.293687 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.294245 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.295378 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.295609 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.295705 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.298514 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:38.298515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:38.298516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:38.298524 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:38.300370 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.300822 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.300823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.300823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.300890 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.301156 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.301157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.301157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.301286 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:38.301366 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.301653 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.301755 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:35:38.302058 31994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.291494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:36351 (local address 127.30.71.2:48765)
0810 06:35:38.291695 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.291741 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.292810 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.293088 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.293092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.293382 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:38.293581 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.293585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.294357 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.294359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.295272 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.295275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.295706 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.295707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.295850 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.296473 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:38.296482 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:38.298414 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:38.300464 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:38.300472 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:38.300475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:38.300726 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:38.300959 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:38.300959 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:38.300960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:38.301086 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:38.301356 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:38.301364 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:38.301517 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.301630 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.301798 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260810 06:35:38.302618 31992 heartbeater.cc:347] Connected to a master server at 127.30.71.62:36351
I20260810 06:35:38.302747 31992 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:38.302938 31992 heartbeater.cc:511] Master 127.30.71.62:36351 requested a full tablet report, sending...
I20260810 06:35:38.303454 31629 ts_manager.cc:194] Registered new tserver with Master: 66fd892d7a184c2090931de9d9c12b33 (127.30.71.2:46271)
I20260810 06:35:38.303855 31629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:48765
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:38.328023 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:36351
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:38.413084 31997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:38.413317 31997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:38.413349 31997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:38.414785 31997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:38.414845 31997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:38.414892 31997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:38.416657 31997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:36351
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:38.416955 31997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:38.417222 31997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:38.422585 32006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:38.422537 32004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:38.422534 32003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:38.422555 31997 server_base.cc:1034] running on GCE node
I20260810 06:35:38.423063 31997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:38.423314 31997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:38.424495 31997 hybrid_clock.cc:648] HybridClock initialized: now 1786343738424471 us; error 39 us; skew 500 ppm
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:38.427735 31997 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:38.428261 31997 webserver.cc:460] Webserver started at http://127.30.71.3:46557/ using document root <none> and password file <none>
I20260810 06:35:38.428578 31997 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:38.428663 31997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:38.428843 31997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:38.430081 31997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "abe76b1c835e4c17b3fb6baebe1693c9"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "76a260af4dd0c115788fe3f57260f830"
server_key_iv: "6cd5cceac2699cd079c9d16582b366fc"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.430485 31997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "abe76b1c835e4c17b3fb6baebe1693c9"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "76a260af4dd0c115788fe3f57260f830"
server_key_iv: "6cd5cceac2699cd079c9d16582b366fc"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.432088 31997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 06:35:38.433002 32013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.433257 31997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:38.433383 31997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "abe76b1c835e4c17b3fb6baebe1693c9"
format_stamp: "Formatted at 2026-08-10 06:35:38 on dist-test-slave-w206"
server_key: "76a260af4dd0c115788fe3f57260f830"
server_key_iv: "6cd5cceac2699cd079c9d16582b366fc"
server_key_version: "encryptionkey@0"
I20260810 06:35:38.433463 31997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:38.452484 31997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:38.455336 31997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:38.455504 31997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:38.455775 31997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:38.456107 31997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:38.456164 31997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.456204 31997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:38.456231 31997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:38.464502 31997 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:37735
I20260810 06:35:38.464551 32139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:37735 every 8 connection(s)
I20260810 06:35:38.464915 31997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:38.474437 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 31997
I20260810 06:35:38.474546 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:38.474749 31004 external_mini_cluster.cc:1428] Setting key 5c884a8567faeb3f52a5c9df584ad21a
I20260810 06:35:38.475564 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.465890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:36289 (local address 127.30.71.62:36351)
0810 06:35:38.465985 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.465988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.466151 (+ 163us) server_negotiation.cc:407] Connection header received
0810 06:35:38.467329 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.467331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.467345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.467393 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:38.468087 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.468596 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.469674 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.469872 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.469948 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.472382 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:38.472384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:38.472384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:38.472391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:38.473767 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.474292 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.474295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.474295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.474366 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.474709 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.474711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.474711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.474865 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:38.474941 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.475350 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.475441 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:35:38.475831 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.465810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:36351 (local address 127.30.71.3:36289)
0810 06:35:38.466009 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.466056 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.467206 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.467464 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.467468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.467764 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:38.467978 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.467983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.468729 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.468730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.469561 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.469563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.469957 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.469958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.470092 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.470638 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:38.470648 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:38.472270 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:38.473856 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:38.473863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:38.473865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:38.474111 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:38.474461 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:38.474462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:38.474463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:38.474652 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:38.474938 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:38.474946 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:38.475176 (+ 230us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.475312 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.475543 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260810 06:35:38.476250 32141 heartbeater.cc:347] Connected to a master server at 127.30.71.62:36351
I20260810 06:35:38.476341 32141 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:38.476548 32141 heartbeater.cc:511] Master 127.30.71.62:36351 requested a full tablet report, sending...
I20260810 06:35:38.477012 31629 ts_manager.cc:194] Registered new tserver with Master: abe76b1c835e4c17b3fb6baebe1693c9 (127.30.71.3:37735)
I20260810 06:35:38.477478 31629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:36289
I20260810 06:35:38.487037 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:35:38.498265 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.489958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.30.71.62:36351)
0810 06:35:38.490014 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.490016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.490161 (+ 145us) server_negotiation.cc:407] Connection header received
0810 06:35:38.490217 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.490254 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.490268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.490329 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:38.491002 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.491999 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.492712 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.493128 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.493210 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.494662 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:38.494663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:38.494664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:38.494674 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:38.496731 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.497208 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.497210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.497210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.497330 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.497570 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.497571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.497572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.497838 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:38.497958 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.498094 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.498173 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:35:38.501768 31629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:38.503813 31629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:38.513180 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.508377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38793 (local address 127.0.0.1:35846)
0810 06:35:38.508862 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.508870 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.508939 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.509211 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.509212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.509229 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:38.509282 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.509283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.510555 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.510556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.511882 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.511890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.512829 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.512830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.512924 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.512942 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.513017 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.513079 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":72,"thread_start_us":91,"threads_started":1}
I20260810 06:35:38.513345 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.508491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35846 (local address 127.30.71.1:38793)
0810 06:35:38.508762 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.508767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.508935 (+ 168us) server_negotiation.cc:407] Connection header received
0810 06:35:38.509009 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.509011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.509031 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.509108 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:38.509392 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.510413 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.511995 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.512720 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.512804 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.512872 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.513049 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.513230 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":124,"threads_started":1}
I20260810 06:35:38.513360 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.508103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:46271 (local address 127.0.0.1:50410)
0810 06:35:38.508566 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.508603 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.508677 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.509065 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.509069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.509071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:38.509154 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.509157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.510593 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.510594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.512180 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.512183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.513138 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.513138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.513196 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.513201 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.513247 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.513290 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":139,"thread_start_us":109,"threads_started":1}
I20260810 06:35:38.513623 32154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.508328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50410 (local address 127.30.71.2:46271)
0810 06:35:38.508910 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.508914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.508922 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:35:38.508956 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.508958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.508970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.509032 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:38.509243 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.510457 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.512278 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.513060 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.513125 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.513185 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.513468 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.513523 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":95,"threads_started":1}
I20260810 06:35:38.514257 31769 tablet_service.cc:1467] Processing CreateTablet for tablet e19aa1e028dc4d668bd5c848721b759b (DEFAULT_TABLE table=test-table [id=d13aa6f8fce845efb8e1f62c2d62551e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:38.514374 31920 tablet_service.cc:1467] Processing CreateTablet for tablet e19aa1e028dc4d668bd5c848721b759b (DEFAULT_TABLE table=test-table [id=d13aa6f8fce845efb8e1f62c2d62551e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:38.514669 31769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e19aa1e028dc4d668bd5c848721b759b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:38.514758 31920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e19aa1e028dc4d668bd5c848721b759b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:38.514761 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.508093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:37735 (local address 127.0.0.1:58534)
0810 06:35:38.508311 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.508697 (+ 386us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.508759 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.509407 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.509408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.509410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:38.509467 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.509469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.510850 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.510851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.512180 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.512183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.514509 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.514511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.514596 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.514600 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.514632 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.514676 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":6,"thread_start_us":82,"threads_started":1}
I20260810 06:35:38.515185 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.508221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.30.71.3:37735)
0810 06:35:38.509165 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.509168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.509175 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:35:38.509232 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.509234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.509248 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.509543 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:38.509630 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.510677 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.513227 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.514396 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.514883 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.514955 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.515010 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.515057 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":229,"threads_started":1}
I20260810 06:35:38.518033 32158 tablet_bootstrap.cc:492] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33: Bootstrap starting.
I20260810 06:35:38.519546 32158 tablet_bootstrap.cc:654] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:38.519905 32067 tablet_service.cc:1467] Processing CreateTablet for tablet e19aa1e028dc4d668bd5c848721b759b (DEFAULT_TABLE table=test-table [id=d13aa6f8fce845efb8e1f62c2d62551e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:38.520399 32067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e19aa1e028dc4d668bd5c848721b759b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:38.520422 32158 log.cc:824] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:38.521289 32158 tablet_bootstrap.cc:492] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33: No bootstrap required, opened a new log
I20260810 06:35:38.521370 32158 ts_tablet_manager.cc:1397] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:35:38.524017 32158 raft_consensus.cc:348] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.524235 32158 raft_consensus.cc:374] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:38.524278 32158 raft_consensus.cc:729] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fd892d7a184c2090931de9d9c12b33, State: Initialized, Role: FOLLOWER
I20260810 06:35:38.524401 32162 tablet_bootstrap.cc:492] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9: Bootstrap starting.
I20260810 06:35:38.524403 32158 consensus_queue.cc:260] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.524678 32160 tablet_bootstrap.cc:492] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339: Bootstrap starting.
I20260810 06:35:38.524712 32158 ts_tablet_manager.cc:1428] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:35:38.524905 31992 heartbeater.cc:503] Master 127.30.71.62:36351 was elected leader, sending a full tablet report...
I20260810 06:35:38.525288 32162 tablet_bootstrap.cc:654] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:38.525488 32160 tablet_bootstrap.cc:654] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:38.526026 32162 log.cc:824] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:38.526293 32160 log.cc:824] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:38.526700 32164 raft_consensus.cc:482] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:38.526811 32164 raft_consensus.cc:504] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.527254 32160 tablet_bootstrap.cc:492] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339: No bootstrap required, opened a new log
I20260810 06:35:38.527325 32160 ts_tablet_manager.cc:1397] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:35:38.528115 32164 leader_election.cc:288] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe76b1c835e4c17b3fb6baebe1693c9 (127.30.71.3:37735), c2d43df9a0084f7f9d3c160987f14339 (127.30.71.1:38793)
I20260810 06:35:38.528824 32162 tablet_bootstrap.cc:492] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9: No bootstrap required, opened a new log
I20260810 06:35:38.528914 32162 ts_tablet_manager.cc:1397] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:35:38.530534 32160 raft_consensus.cc:348] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.530735 32160 raft_consensus.cc:374] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:38.530777 32160 raft_consensus.cc:729] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d43df9a0084f7f9d3c160987f14339, State: Initialized, Role: FOLLOWER
I20260810 06:35:38.530946 32160 consensus_queue.cc:260] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.531298 32160 ts_tablet_manager.cc:1428] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260810 06:35:38.531419 31841 heartbeater.cc:503] Master 127.30.71.62:36351 was elected leader, sending a full tablet report...
I20260810 06:35:38.531293 32162 raft_consensus.cc:348] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.531491 32162 raft_consensus.cc:374] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:38.531530 32162 raft_consensus.cc:729] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe76b1c835e4c17b3fb6baebe1693c9, State: Initialized, Role: FOLLOWER
I20260810 06:35:38.531651 32162 consensus_queue.cc:260] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.532019 32162 ts_tablet_manager.cc:1428] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:35:38.532188 32141 heartbeater.cc:503] Master 127.30.71.62:36351 was elected leader, sending a full tablet report...
I20260810 06:35:38.532253 31994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.528417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38793 (local address 127.30.71.2:60753)
0810 06:35:38.528611 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.528621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.528675 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.528843 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.528844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.528845 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:38.528941 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.528944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.529616 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.529617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.530852 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.530855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.531964 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.531965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.532064 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.532068 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.532120 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.532168 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":112,"threads_started":1}
I20260810 06:35:38.532599 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.528769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:57845 (local address 127.30.71.3:37735)
0810 06:35:38.528862 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.528864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.528874 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:38.528897 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.528898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.528908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.528962 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:38.529275 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.529972 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.531486 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.532176 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.532225 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.532237 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.532498 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.532540 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:35:38.532707 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.528481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:60753 (local address 127.30.71.1:38793)
0810 06:35:38.528526 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.528529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.528660 (+ 131us) server_negotiation.cc:407] Connection header received
0810 06:35:38.528734 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.528735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.528746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.528798 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:38.529022 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.529514 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.530958 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.531632 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.531716 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.531730 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.532596 (+ 866us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.532642 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:35:38.532827 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.528290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:37735 (local address 127.30.71.2:57845)
0810 06:35:38.528585 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:38.528598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:38.528659 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:38.529030 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:38.529031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:38.529034 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:38.529163 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.529165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.530087 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.530088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.531354 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:38.531357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.532315 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.532316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:38.532389 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.532392 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:38.532687 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 06:35:38.532739 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260810 06:35:38.533036 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19aa1e028dc4d668bd5c848721b759b" candidate_uuid: "66fd892d7a184c2090931de9d9c12b33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d43df9a0084f7f9d3c160987f14339" is_pre_election: true
I20260810 06:35:38.533179 31791 raft_consensus.cc:2454] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fd892d7a184c2090931de9d9c12b33 in term 0.
I20260810 06:35:38.533594 31867 leader_election.cc:302] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fd892d7a184c2090931de9d9c12b33, c2d43df9a0084f7f9d3c160987f14339; no voters:
I20260810 06:35:38.533746 32164 raft_consensus.cc:2781] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:38.533806 32164 raft_consensus.cc:482] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:38.533833 32164 raft_consensus.cc:3037] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:38.533983 32088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19aa1e028dc4d668bd5c848721b759b" candidate_uuid: "66fd892d7a184c2090931de9d9c12b33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" is_pre_election: true
I20260810 06:35:38.534111 32088 raft_consensus.cc:2454] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fd892d7a184c2090931de9d9c12b33 in term 0.
I20260810 06:35:38.535077 32164 raft_consensus.cc:504] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.535210 32164 leader_election.cc:288] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [CANDIDATE]: Term 1 election: Requested vote from peers abe76b1c835e4c17b3fb6baebe1693c9 (127.30.71.3:37735), c2d43df9a0084f7f9d3c160987f14339 (127.30.71.1:38793)
I20260810 06:35:38.535388 32088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19aa1e028dc4d668bd5c848721b759b" candidate_uuid: "66fd892d7a184c2090931de9d9c12b33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe76b1c835e4c17b3fb6baebe1693c9"
I20260810 06:35:38.535389 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19aa1e028dc4d668bd5c848721b759b" candidate_uuid: "66fd892d7a184c2090931de9d9c12b33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d43df9a0084f7f9d3c160987f14339"
I20260810 06:35:38.535465 32088 raft_consensus.cc:3037] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:38.535465 31791 raft_consensus.cc:3037] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:38.536419 31791 raft_consensus.cc:2454] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fd892d7a184c2090931de9d9c12b33 in term 1.
I20260810 06:35:38.536439 32088 raft_consensus.cc:2454] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fd892d7a184c2090931de9d9c12b33 in term 1.
I20260810 06:35:38.536582 31867 leader_election.cc:302] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fd892d7a184c2090931de9d9c12b33, abe76b1c835e4c17b3fb6baebe1693c9; no voters:
I20260810 06:35:38.536705 32164 raft_consensus.cc:2781] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:38.536897 32164 raft_consensus.cc:686] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [term 1 LEADER]: Becoming Leader. State: Replica: 66fd892d7a184c2090931de9d9c12b33, State: Running, Role: LEADER
I20260810 06:35:38.536996 32164 consensus_queue.cc:237] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } }
I20260810 06:35:38.537992 31629 catalog_manager.cc:5495] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66fd892d7a184c2090931de9d9c12b33 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "66fd892d7a184c2090931de9d9c12b33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe76b1c835e4c17b3fb6baebe1693c9" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2d43df9a0084f7f9d3c160987f14339" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66fd892d7a184c2090931de9d9c12b33" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46271 } health_report { overall_health: HEALTHY } } }
W20260810 06:35:38.541016 31993 tablet.cc:2369] T e19aa1e028dc4d668bd5c848721b759b P 66fd892d7a184c2090931de9d9c12b33: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:35:38.548481 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.540367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.30.71.1:38793)
0810 06:35:38.540410 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.540412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.540698 (+ 286us) server_negotiation.cc:407] Connection header received
0810 06:35:38.540796 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.540797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.540807 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.540859 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:38.541456 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.541926 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.542778 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.543017 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.543082 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.545560 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:38.545572 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:38.545572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:38.545581 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:38.547455 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.547810 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.547812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.547813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.547871 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.548048 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.548048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.548049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.548184 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:38.548317 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.548348 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.548411 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260810 06:35:38.618898 31842 tablet.cc:2369] T e19aa1e028dc4d668bd5c848721b759b P c2d43df9a0084f7f9d3c160987f14339: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:35:38.624707 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.614945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.30.71.1:38793)
0810 06:35:38.615025 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.615028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.615158 (+ 130us) server_negotiation.cc:407] Connection header received
0810 06:35:38.615302 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.615303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.615315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.615384 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:38.616042 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.616878 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.617708 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.618082 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.619036 (+ 954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.621679 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:38.621681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:38.621681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:38.621691 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:38.623439 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.623822 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.623824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.623824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.623885 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.624104 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.624105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.624106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.624275 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:38.624376 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.624566 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.624638 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260810 06:35:38.715651 32142 tablet.cc:2369] T e19aa1e028dc4d668bd5c848721b759b P abe76b1c835e4c17b3fb6baebe1693c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:35:38 dist-test-slave-w206 krb5kdc[31580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:35:38.723215 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:38.714965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.30.71.1:38793)
0810 06:35:38.715061 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:38.715063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:38.715146 (+ 83us) server_negotiation.cc:407] Connection header received
0810 06:35:38.715267 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:38.715268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:38.715282 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:38.715364 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:38.716057 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.716563 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.717380 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:38.717579 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:38.717672 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:38.720393 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:38.720395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:38.720395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:38.720404 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:38.721826 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.722295 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.722296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.722297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.722360 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:38.722590 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:38.722591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:38.722591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:38.722714 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:38.722833 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:38.722961 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:35:38.723085 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260810 06:35:38.723980 31746 tablet_service.cc:2908] Rejecting scan request for tablet e19aa1e028dc4d668bd5c848721b759b: Uninitialized: clean time has not yet been initialized
I20260810 06:35:38.725165 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31696
I20260810 06:35:38.732280 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31847
I20260810 06:35:38.738868 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31997
I20260810 06:35:38.745675 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 31593
2026-08-10T06:35:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2277 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:40 dist-test-slave-w206 krb5kdc[32186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:40 dist-test-slave-w206 krb5kdc[32186](info): set up 2 sockets
Aug 10 06:35:40 dist-test-slave-w206 krb5kdc[32186](info): commencing operation
krb5kdc: starting...
W20260810 06:35:41.227064 31004 mini_kdc.cc:121] Time spent starting KDC: real 2.458s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:41 dist-test-slave-w206 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:41.358429 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:35355
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:35355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:41.444608 32199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:41.444873 32199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:41.444913 32199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:41.446070 32199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:41.446141 32199 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:35:41.446164 32199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:41.446193 32199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:41.446221 32199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:41.446251 32199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:41.447817 32199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:35355
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:35355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:41.448107 32199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:41.448344 32199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:41.453456 32208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:41.453466 32206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:41.453620 32199 server_base.cc:1034] running on GCE node
W20260810 06:35:41.453450 32205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:41.453963 32199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:41.454273 32199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:41.455448 32199 hybrid_clock.cc:648] HybridClock initialized: now 1786343741455423 us; error 45 us; skew 500 ppm
Aug 10 06:35:41 dist-test-slave-w206 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:41.458248 32199 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:41.458747 32199 webserver.cc:460] Webserver started at http://127.30.71.62:41371/ using document root <none> and password file <none>
I20260810 06:35:41.459100 32199 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:41.459173 32199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:41.459313 32199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:41.460474 32199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "4e3dd7322ad7475597d4090b78c928c2"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "e42f35279b7c15ff5e47799c5c53fb83"
server_key_iv: "f217f35cfd81de22a9d8472a62d6d7b7"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.460866 32199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "4e3dd7322ad7475597d4090b78c928c2"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "e42f35279b7c15ff5e47799c5c53fb83"
server_key_iv: "f217f35cfd81de22a9d8472a62d6d7b7"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.462399 32199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:41.463208 32216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:41.463374 32199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:41.463487 32199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "4e3dd7322ad7475597d4090b78c928c2"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "e42f35279b7c15ff5e47799c5c53fb83"
server_key_iv: "f217f35cfd81de22a9d8472a62d6d7b7"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.463563 32199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:41.491415 32199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:41.494131 32199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:41.494318 32199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:41.499584 32199 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:35355
I20260810 06:35:41.499639 32287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:35355 every 8 connection(s)
I20260810 06:35:41.500011 32199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:41.500665 32288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:41.503233 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2: Bootstrap starting.
I20260810 06:35:41.504000 32288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:41.504314 32288 log.cc:824] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:41.504514 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 32199
I20260810 06:35:41.504654 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:41.504849 31004 external_mini_cluster.cc:1428] Setting key ce051f0db1563fd5746d53b67679d1a9
I20260810 06:35:41.505211 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2: No bootstrap required, opened a new log
I20260810 06:35:41.507813 32288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3dd7322ad7475597d4090b78c928c2" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35355 } }
I20260810 06:35:41.507985 32288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:41.508050 32288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3dd7322ad7475597d4090b78c928c2, State: Initialized, Role: FOLLOWER
I20260810 06:35:41.508212 32288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3dd7322ad7475597d4090b78c928c2" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35355 } }
I20260810 06:35:41.508319 32288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:41.508363 32288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:41.508417 32288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:41.508992 32288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3dd7322ad7475597d4090b78c928c2" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35355 } }
I20260810 06:35:41.509111 32288 leader_election.cc:302] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e3dd7322ad7475597d4090b78c928c2; no voters:
I20260810 06:35:41.509318 32288 leader_election.cc:288] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:41.509387 32293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:41.509534 32293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [term 1 LEADER]: Becoming Leader. State: Replica: 4e3dd7322ad7475597d4090b78c928c2, State: Running, Role: LEADER
I20260810 06:35:41.509593 32288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:41.509711 32293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3dd7322ad7475597d4090b78c928c2" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35355 } }
I20260810 06:35:41.510087 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e3dd7322ad7475597d4090b78c928c2. Latest consensus state: current_term: 1 leader_uuid: "4e3dd7322ad7475597d4090b78c928c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3dd7322ad7475597d4090b78c928c2" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35355 } } }
I20260810 06:35:41.510185 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:41.510443 32294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e3dd7322ad7475597d4090b78c928c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3dd7322ad7475597d4090b78c928c2" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35355 } } }
I20260810 06:35:41.510771 32294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:41.510919 32300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:41.511512 32300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:41.513203 32300 catalog_manager.cc:1347] Generated new cluster ID: 20ca7f979f3a4526869f3177130bfa8b
I20260810 06:35:41.513259 32300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:35:41 dist-test-slave-w206 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:41.518144 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.505366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54470 (local address 127.30.71.62:35355)
0810 06:35:41.505542 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:41.505553 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:41.505588 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:35:41.507799 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:41.507806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:41.507880 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:41.508045 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:41.508806 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.510099 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.511279 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.512013 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.512152 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.514732 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:41.514736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:41.514737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:41.514748 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:41.516504 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.516914 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.516917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.516918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.517035 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.517260 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.517261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.517261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.517515 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:41.517703 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:41.517775 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 06:35:41.517945 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260810 06:35:41.520494 32300 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:41.521023 32300 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:41.523908 32300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2: Generated new TSK 0
I20260810 06:35:41.524096 32300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:41.554411 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35355
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:41.653439 32315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:41.653859 32315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:41.653924 32315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:41.656188 32315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:41.656271 32315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:41.656314 32315 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:35:41.656409 32315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:41.659192 32315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:41.659513 32315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:41.659838 32315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:41.666376 32325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:41.666352 32321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:41.666639 32322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:41.667111 32315 server_base.cc:1034] running on GCE node
I20260810 06:35:41.667388 32315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:41.667726 32315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:41.668933 32315 hybrid_clock.cc:648] HybridClock initialized: now 1786343741668910 us; error 35 us; skew 500 ppm
Aug 10 06:35:41 dist-test-slave-w206 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:41.672374 32315 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:41.672930 32315 webserver.cc:460] Webserver started at http://127.30.71.1:45915/ using document root <none> and password file <none>
I20260810 06:35:41.673219 32315 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:41.673296 32315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:41.673465 32315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:41.674595 32315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "c2371e46865d4337bd68cb9d69092c1c"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "a4eacaf8d4e50fee0bc7db06a5e9a099"
server_key_iv: "0038a6e7c84e1af42b7a5933b73f39e6"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.675064 32315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "c2371e46865d4337bd68cb9d69092c1c"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "a4eacaf8d4e50fee0bc7db06a5e9a099"
server_key_iv: "0038a6e7c84e1af42b7a5933b73f39e6"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.676638 32315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:41.677511 32333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:41.677700 32315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:41.677807 32315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "c2371e46865d4337bd68cb9d69092c1c"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "a4eacaf8d4e50fee0bc7db06a5e9a099"
server_key_iv: "0038a6e7c84e1af42b7a5933b73f39e6"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.677896 32315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:41.690249 32315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:41.692768 32315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:41.692940 32315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:41.693629 32315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:41.693732 32315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:41.693805 32315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:41.693853 32315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:41.701097 32315 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:40655
I20260810 06:35:41.701195 32459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:40655 every 8 connection(s)
Aug 10 06:35:41 dist-test-slave-w206 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:41.702306 32315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:41.705410 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.694194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:54825 (local address 127.30.71.62:35355)
0810 06:35:41.694262 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:41.694265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:41.694549 (+ 284us) server_negotiation.cc:407] Connection header received
0810 06:35:41.695235 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:41.695237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:41.695266 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:41.695347 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:41.696330 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.697171 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.698905 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.699385 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.699465 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.702270 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:41.702271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:41.702271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:41.702279 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:41.703944 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.704396 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.704398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.704398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.704467 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.704728 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.704729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.704729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.704857 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:41.704939 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:41.705218 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 06:35:41.705321 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:35:41.705588 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.694191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35355 (local address 127.30.71.1:54825)
0810 06:35:41.694386 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:41.694431 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:41.695068 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:41.695449 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:41.695454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:41.695872 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:41.696207 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.696214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.697305 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.697307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.698796 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.698799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.699546 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.699547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.699732 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.700383 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:41.700395 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:41.702153 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:41.704055 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.704062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.704065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.704300 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.704539 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.704540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.704540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.704656 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.704930 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:41.704937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:41.705081 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 06:35:41.705193 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:35:41.705356 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":122,"threads_started":1}
I20260810 06:35:41.706127 32460 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35355
I20260810 06:35:41.706243 32460 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:41.706460 32460 heartbeater.cc:511] Master 127.30.71.62:35355 requested a full tablet report, sending...
I20260810 06:35:41.707248 32234 ts_manager.cc:194] Registered new tserver with Master: c2371e46865d4337bd68cb9d69092c1c (127.30.71.1:40655)
I20260810 06:35:41.708218 32234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:54825
I20260810 06:35:41.710920 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 32315
I20260810 06:35:41.711052 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:41.711264 31004 external_mini_cluster.cc:1428] Setting key 8ec0e0d2fecf25c421edf12c8fc38ab3
I20260810 06:35:41.714694 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.707573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:47881 (local address 127.30.71.62:35355)
0810 06:35:41.707619 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:41.707621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:41.707724 (+ 103us) server_negotiation.cc:407] Connection header received
0810 06:35:41.707758 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:41.707759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:41.707770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:41.707845 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:41.708192 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.709121 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.709816 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.710224 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.710303 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.711617 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:41.711618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:41.711619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:41.711625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:41.713423 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.713842 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.713843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.713844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.713907 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.714136 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.714137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.714138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.714251 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:41.714334 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:41.714519 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 06:35:41.714601 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":23}
I20260810 06:35:41.714756 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.707500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35355 (local address 127.30.71.1:47881)
0810 06:35:41.707648 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:41.707658 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:41.707716 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:41.707849 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:41.707850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:41.708038 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:41.708106 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.708108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.709207 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.709208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.709730 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.709732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.710309 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.710310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.710412 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.710956 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:41.710961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:41.711473 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:41.713522 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.713523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.713524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.713758 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.713988 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.713989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.713989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.714056 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.714339 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:41.714340 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:41.714401 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 06:35:41.714487 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:35:41.714602 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:41.747239 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35355
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:41.831121 32470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:41.831403 32470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:41.831449 32470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:41.832683 32470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:41.832767 32470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:41.832806 32470 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:35:41.832862 32470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:41.834369 32470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:41.834664 32470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:41.834918 32470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:41.840319 32476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:41.840389 32479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:41.840474 32470 server_base.cc:1034] running on GCE node
W20260810 06:35:41.840404 32477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:41.840824 32470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:41.841086 32470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:41.842281 32470 hybrid_clock.cc:648] HybridClock initialized: now 1786343741842263 us; error 42 us; skew 500 ppm
Aug 10 06:35:41 dist-test-slave-w206 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:41.844910 32470 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:41.845470 32470 webserver.cc:460] Webserver started at http://127.30.71.2:44503/ using document root <none> and password file <none>
I20260810 06:35:41.845741 32470 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:41.845816 32470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:41.845971 32470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:41.847263 32470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "edcffcd503a14f30a78a5289a92b5bc7"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "b91d81e795a46cdb25c6607e31f4e0b2"
server_key_iv: "12a0be21528bb5f4cc168be798bf6094"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.847658 32470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "edcffcd503a14f30a78a5289a92b5bc7"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "b91d81e795a46cdb25c6607e31f4e0b2"
server_key_iv: "12a0be21528bb5f4cc168be798bf6094"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.849058 32470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:35:41.849833 32487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:41.850035 32470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:41.850129 32470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "edcffcd503a14f30a78a5289a92b5bc7"
format_stamp: "Formatted at 2026-08-10 06:35:41 on dist-test-slave-w206"
server_key: "b91d81e795a46cdb25c6607e31f4e0b2"
server_key_iv: "12a0be21528bb5f4cc168be798bf6094"
server_key_version: "encryptionkey@0"
I20260810 06:35:41.850214 32470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:41.869843 32470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:41.871975 32470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:41.872146 32470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:41.872761 32470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:41.872823 32470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:41.872867 32470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:41.872895 32470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:41.880514 32470 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:34347
I20260810 06:35:41.880589 32614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:34347 every 8 connection(s)
I20260810 06:35:41.880995 32470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:35:41 dist-test-slave-w206 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:41.883571 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 32470
I20260810 06:35:41.883653 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:41.883888 31004 external_mini_cluster.cc:1428] Setting key 9337abcdbf8e46f10fec4a541bdeca98
I20260810 06:35:41.884928 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.873282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:37465 (local address 127.30.71.62:35355)
0810 06:35:41.873342 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:41.873344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:41.873657 (+ 313us) server_negotiation.cc:407] Connection header received
0810 06:35:41.874299 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:41.874301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:41.874316 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:41.874406 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:41.875353 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.876198 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.877965 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.878287 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.878375 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.881653 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:41.881655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:41.881656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:41.881666 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:41.883308 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.883802 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.883804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.883805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.883868 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.884146 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.884147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.884147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.884301 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:41.884370 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:41.884719 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 06:35:41.884818 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:35:41.885028 32496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.873287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35355 (local address 127.30.71.2:37465)
0810 06:35:41.873488 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:41.873534 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:41.874119 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:41.874524 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:41.874530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:41.875018 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:41.875269 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.875274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.876345 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.876346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.877878 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.877882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.878475 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.878476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.878646 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.879465 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:41.879476 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:41.881535 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:41.883427 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.883434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.883436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.883726 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.883935 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.883936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.883937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.884074 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.884361 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:41.884367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:41.884523 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 06:35:41.884650 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:35:41.884822 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":126,"threads_started":1}
I20260810 06:35:41.886629 32615 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35355
I20260810 06:35:41.886770 32615 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:41.887006 32615 heartbeater.cc:511] Master 127.30.71.62:35355 requested a full tablet report, sending...
I20260810 06:35:41.887415 32235 ts_manager.cc:194] Registered new tserver with Master: edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2:34347)
I20260810 06:35:41.887900 32235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:37465
I20260810 06:35:41.893496 32622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.886674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35355 (local address 127.30.71.2:42459)
0810 06:35:41.886943 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:41.886953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:41.887036 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:41.887235 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:41.887236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:41.887444 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:41.887532 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.887535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.888459 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.888460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.889145 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:41.889147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.889636 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.889637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:41.889692 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.890255 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:41.890259 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:41.890567 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:41.892424 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.892425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.892425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.892599 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.892820 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:41.892820 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:41.892820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:41.892874 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:41.893238 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:41.893239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:41.893292 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:35:41.893342 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:35:41.893421 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":85,"threads_started":1}
I20260810 06:35:41.893582 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:41.886869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:42459 (local address 127.30.71.62:35355)
0810 06:35:41.886938 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:41.886940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:41.887022 (+ 82us) server_negotiation.cc:407] Connection header received
0810 06:35:41.887130 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:41.887130 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:41.887142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:41.887212 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:41.887610 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.888358 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.889235 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:41.889559 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:41.889639 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:41.890659 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:41.890661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:41.890661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:41.890669 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:41.892331 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.892678 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.892680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.892680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.892752 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:41.892945 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:41.892946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:41.892947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:41.893158 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:41.893234 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:41.893398 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:35:41.893488 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:41.914887 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35355
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:41.999850 32624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:42.000196 32624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:42.000227 32624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:42.001518 32624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:42.001564 32624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:42.001586 32624 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:35:42.001605 32624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:42.003109 32624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:42.003315 32624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:42.003512 32624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:42.008798 32630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:42.008781 32632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:42.008867 32624 server_base.cc:1034] running on GCE node
W20260810 06:35:42.008782 32629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:42.009232 32624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:42.009557 32624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:42.010725 32624 hybrid_clock.cc:648] HybridClock initialized: now 1786343742010716 us; error 42 us; skew 500 ppm
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343742, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:42.013634 32624 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:42.014127 32624 webserver.cc:460] Webserver started at http://127.30.71.3:37593/ using document root <none> and password file <none>
I20260810 06:35:42.014406 32624 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:42.014483 32624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:42.014645 32624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:42.015823 32624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2"
format_stamp: "Formatted at 2026-08-10 06:35:42 on dist-test-slave-w206"
server_key: "f827b5e1ee35bc6996677cf958080f02"
server_key_iv: "8956a5b17da7b81d3ab691359a929f97"
server_key_version: "encryptionkey@0"
I20260810 06:35:42.016206 32624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2"
format_stamp: "Formatted at 2026-08-10 06:35:42 on dist-test-slave-w206"
server_key: "f827b5e1ee35bc6996677cf958080f02"
server_key_iv: "8956a5b17da7b81d3ab691359a929f97"
server_key_version: "encryptionkey@0"
I20260810 06:35:42.017777 32624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:35:42.018662 32640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:42.018869 32624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:42.018980 32624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2"
format_stamp: "Formatted at 2026-08-10 06:35:42 on dist-test-slave-w206"
server_key: "f827b5e1ee35bc6996677cf958080f02"
server_key_iv: "8956a5b17da7b81d3ab691359a929f97"
server_key_version: "encryptionkey@0"
I20260810 06:35:42.019098 32624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:42.036288 32624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:42.038362 32624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:42.038547 32624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:42.039242 32624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:42.039307 32624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:42.039352 32624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:42.039381 32624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343742, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:42.047653 32624 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:45345
I20260810 06:35:42.047884 305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:45345 every 8 connection(s)
I20260810 06:35:42.048142 32624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:35:42.050249 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.039639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:55923 (local address 127.30.71.62:35355)
0810 06:35:42.039722 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.039725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.039983 (+ 258us) server_negotiation.cc:407] Connection header received
0810 06:35:42.040468 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.040470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.040486 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.040568 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:42.041389 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.042196 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.043989 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.044320 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.044401 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.047251 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:42.047253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:42.047253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:42.047273 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:42.049034 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.049411 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.049412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.049412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.049468 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.049684 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.049684 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.049684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.049793 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:42.049845 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.050110 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.050184 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:35:42.050518 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.039628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35355 (local address 127.30.71.3:55923)
0810 06:35:42.039822 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.039865 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.040364 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.040718 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.040723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.041079 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:42.041277 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.041283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.042315 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.042316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.043885 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.043888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.044447 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.044448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.044612 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.045242 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:42.045263 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:42.047096 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:42.049110 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:42.049115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:42.049116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:42.049333 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:42.049529 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:42.049530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:42.049530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:42.049629 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:42.049870 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:42.049875 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:42.049996 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.050080 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.050271 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260810 06:35:42.050889 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 32624
I20260810 06:35:42.050997 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:42.051077 306 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35355
I20260810 06:35:42.051164 306 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:42.051200 31004 external_mini_cluster.cc:1428] Setting key d20d9fcbc41f9643bc4d56d372222528
I20260810 06:35:42.051375 306 heartbeater.cc:511] Master 127.30.71.62:35355 requested a full tablet report, sending...
I20260810 06:35:42.051738 32234 ts_manager.cc:194] Registered new tserver with Master: d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345)
I20260810 06:35:42.052157 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:35:42.052189 32234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:55923
I20260810 06:35:42.058957 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.053048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35355 (local address 127.30.71.3:35237)
0810 06:35:42.053235 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.053247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.053322 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.053454 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.053455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.053636 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:42.053696 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.053698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.054502 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.054503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.055082 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.055084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.055466 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.055467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.055548 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.055967 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:42.055971 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:42.056298 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:42.057856 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:42.057857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:42.057858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:42.058035 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:42.058240 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:42.058241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:42.058242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:42.058315 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:42.058566 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:42.058567 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:42.058615 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.058714 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.058830 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":133,"threads_started":1}
I20260810 06:35:42.059065 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.053107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:35237 (local address 127.30.71.62:35355)
0810 06:35:42.053150 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.053152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.053307 (+ 155us) server_negotiation.cc:407] Connection header received
0810 06:35:42.053381 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.053382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.053392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.053449 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:42.053784 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.054400 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.055176 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.055380 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.055464 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.056395 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:42.056396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:42.056397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:42.056405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:42.057740 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.058109 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.058110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.058111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.058172 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.058386 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.058387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.058387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.058499 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:42.058574 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.058742 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.058947 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:42.078263 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.070573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54472 (local address 127.30.71.62:35355)
0810 06:35:42.070652 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.070655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.070675 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:35:42.070736 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.070737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.070749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.070812 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:42.071627 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.072234 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.072956 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.073171 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.073282 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.075881 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:42.075882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:42.075883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:42.075890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:42.077240 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.077568 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.077569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.077569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.077621 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.077792 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.077792 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.077793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.077892 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:42.077966 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.078066 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.078126 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:35:42.080120 32234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:42.081854 32234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:42.089071 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.084629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.30.71.3:45345)
0810 06:35:42.085206 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.085211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.085221 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:42.085258 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.085260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.085275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.085370 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:42.085491 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.086661 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.087808 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.088575 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.088659 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.088724 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.088915 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.088959 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":98,"threads_started":1}
I20260810 06:35:42.089063 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.084546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:40655 (local address 127.0.0.1:45022)
0810 06:35:42.085455 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.085465 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.085514 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.085713 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.085713 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.085715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:42.085748 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.085749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.086663 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.086664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.087837 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.087840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.088701 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.088702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.088791 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.088807 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.088887 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.088956 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":27,"thread_start_us":81,"threads_started":1}
I20260810 06:35:42.089062 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.084316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:45345 (local address 127.0.0.1:44890)
0810 06:35:42.084491 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.084534 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.084604 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.085367 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.085368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.085370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:42.085407 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.085426 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.086746 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.086747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.087706 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.087708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.088685 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.088686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.088791 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.088807 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.088887 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.088956 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":5,"thread_start_us":67,"threads_started":1}
I20260810 06:35:42.089816 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.084660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.30.71.1:40655)
0810 06:35:42.084838 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.084843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.085574 (+ 731us) server_negotiation.cc:407] Connection header received
0810 06:35:42.085617 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.085619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.085630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.085718 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:42.085805 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.086557 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.087925 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.088614 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.088679 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.088725 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.089641 (+ 916us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.089688 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260810 06:35:42.089916 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.084319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:34347 (local address 127.0.0.1:41656)
0810 06:35:42.084766 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.084776 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.084814 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.085074 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.085076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.085079 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:42.085150 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.085151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.086488 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.086489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.087838 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.087840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.089644 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.089646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.089744 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.089748 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.089809 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.089852 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":95,"threads_started":1}
I20260810 06:35:42.089990 325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.084429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.30.71.2:34347)
0810 06:35:42.084904 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.084909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.084926 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:35:42.084959 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.084961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.084972 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.085064 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:42.085260 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.086397 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.088791 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.089552 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.089632 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.089691 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.089840 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.089885 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":84,"threads_started":1}
I20260810 06:35:42.090258 32700 tablet_service.cc:1467] Processing CreateTablet for tablet 21a693610e9744f493d007042c9aaa2c (DEFAULT_TABLE table=test-table [id=80bd672141da4491a3f4ac0d0f664d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:42.090647 32700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a693610e9744f493d007042c9aaa2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.090626 32541 tablet_service.cc:1467] Processing CreateTablet for tablet 21a693610e9744f493d007042c9aaa2c (DEFAULT_TABLE table=test-table [id=80bd672141da4491a3f4ac0d0f664d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:42.090808 32390 tablet_service.cc:1467] Processing CreateTablet for tablet 21a693610e9744f493d007042c9aaa2c (DEFAULT_TABLE table=test-table [id=80bd672141da4491a3f4ac0d0f664d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:42.091037 32541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a693610e9744f493d007042c9aaa2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.091195 32390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a693610e9744f493d007042c9aaa2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.094244 329 tablet_bootstrap.cc:492] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Bootstrap starting.
I20260810 06:35:42.094467 330 tablet_bootstrap.cc:492] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Bootstrap starting.
I20260810 06:35:42.094630 331 tablet_bootstrap.cc:492] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Bootstrap starting.
I20260810 06:35:42.095275 329 tablet_bootstrap.cc:654] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.095312 331 tablet_bootstrap.cc:654] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.095672 331 log.cc:824] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:42.095700 330 tablet_bootstrap.cc:654] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.095697 329 log.cc:824] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:42.096064 330 log.cc:824] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:42.096709 331 tablet_bootstrap.cc:492] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: No bootstrap required, opened a new log
I20260810 06:35:42.096832 330 tablet_bootstrap.cc:492] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: No bootstrap required, opened a new log
I20260810 06:35:42.096823 331 ts_tablet_manager.cc:1397] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:35:42.096832 329 tablet_bootstrap.cc:492] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: No bootstrap required, opened a new log
I20260810 06:35:42.096895 330 ts_tablet_manager.cc:1397] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:35:42.096899 329 ts_tablet_manager.cc:1397] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:35:42.098290 329 raft_consensus.cc:348] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.098461 329 raft_consensus.cc:374] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.098507 329 raft_consensus.cc:729] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2371e46865d4337bd68cb9d69092c1c, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.098623 329 consensus_queue.cc:260] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.098924 329 ts_tablet_manager.cc:1428] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:35:42.099047 32460 heartbeater.cc:503] Master 127.30.71.62:35355 was elected leader, sending a full tablet report...
I20260810 06:35:42.098903 331 raft_consensus.cc:348] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.098903 330 raft_consensus.cc:348] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.099090 331 raft_consensus.cc:374] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.099123 330 raft_consensus.cc:374] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.099131 331 raft_consensus.cc:729] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edcffcd503a14f30a78a5289a92b5bc7, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.099160 330 raft_consensus.cc:729] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d673bd4b5bd44e8c86e2b0a5b7b4ade2, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.099249 331 consensus_queue.cc:260] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.099291 330 consensus_queue.cc:260] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.099520 331 ts_tablet_manager.cc:1428] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:35:42.099520 330 ts_tablet_manager.cc:1428] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:35:42.099642 32615 heartbeater.cc:503] Master 127.30.71.62:35355 was elected leader, sending a full tablet report...
I20260810 06:35:42.099936 306 heartbeater.cc:503] Master 127.30.71.62:35355 was elected leader, sending a full tablet report...
W20260810 06:35:42.131903 32616 tablet.cc:2369] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:35:42.203266 32461 tablet.cc:2369] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:35:42.203680 335 raft_consensus.cc:482] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:42.203814 335 raft_consensus.cc:504] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.204331 335 leader_election.cc:288] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345), edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2:34347)
I20260810 06:35:42.207621 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.204791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:37475 (local address 127.30.71.3:45345)
0810 06:35:42.204846 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.204849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.204875 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:35:42.204916 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.204917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.204928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.205005 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:42.205296 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.205785 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.206813 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.207239 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.207340 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.207350 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.207513 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.207553 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:35:42.207621 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.204645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:45345 (local address 127.30.71.1:37475)
0810 06:35:42.204726 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.204741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.204813 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.205053 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.205054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.205056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:42.205127 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.205131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.205882 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.205883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.206720 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.206723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.207350 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.207351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.207437 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.207441 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.207507 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.207550 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:35:42.208097 32722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a693610e9744f493d007042c9aaa2c" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" is_pre_election: true
I20260810 06:35:42.208340 340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.204645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:34347 (local address 127.30.71.1:32997)
0810 06:35:42.204886 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.204898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.204967 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.205120 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.205121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.205124 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:42.205217 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.205221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.205946 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.205947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.206874 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.206877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.207989 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.207992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.208084 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.208090 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.208179 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.208224 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":93,"thread_start_us":68,"threads_started":1}
I20260810 06:35:42.208328 32722 raft_consensus.cc:2454] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 0.
I20260810 06:35:42.208344 325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.204767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:32997 (local address 127.30.71.2:34347)
0810 06:35:42.204815 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.204818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.204984 (+ 166us) server_negotiation.cc:407] Connection header received
0810 06:35:42.205021 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.205023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.205038 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.205099 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:42.205320 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.205845 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.207025 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.207531 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.207640 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.207652 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.208205 (+ 553us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.208275 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:35:42.208658 32338 leader_election.cc:302] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2371e46865d4337bd68cb9d69092c1c, d673bd4b5bd44e8c86e2b0a5b7b4ade2; no voters:
I20260810 06:35:42.208702 32543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a693610e9744f493d007042c9aaa2c" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edcffcd503a14f30a78a5289a92b5bc7" is_pre_election: true
I20260810 06:35:42.208828 335 raft_consensus.cc:2781] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:42.208854 32543 raft_consensus.cc:2454] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 0.
I20260810 06:35:42.208916 335 raft_consensus.cc:482] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:42.208958 335 raft_consensus.cc:3037] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.210295 335 raft_consensus.cc:504] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.210486 335 leader_election.cc:288] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 election: Requested vote from peers d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345), edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2:34347)
I20260810 06:35:42.210692 32722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a693610e9744f493d007042c9aaa2c" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2"
I20260810 06:35:42.210858 32722 raft_consensus.cc:3037] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.211961 32722 raft_consensus.cc:2454] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 1.
I20260810 06:35:42.212131 32543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a693610e9744f493d007042c9aaa2c" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edcffcd503a14f30a78a5289a92b5bc7"
I20260810 06:35:42.212184 32338 leader_election.cc:302] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2371e46865d4337bd68cb9d69092c1c, d673bd4b5bd44e8c86e2b0a5b7b4ade2; no voters:
I20260810 06:35:42.212248 32543 raft_consensus.cc:3037] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.212317 335 raft_consensus.cc:2781] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:42.212508 335 raft_consensus.cc:686] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 1 LEADER]: Becoming Leader. State: Replica: c2371e46865d4337bd68cb9d69092c1c, State: Running, Role: LEADER
I20260810 06:35:42.212617 335 consensus_queue.cc:237] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } }
I20260810 06:35:42.213454 32543 raft_consensus.cc:2454] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 1.
I20260810 06:35:42.213692 32234 catalog_manager.cc:5495] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c reported cstate change: term changed from 0 to 1, leader changed from <none> to c2371e46865d4337bd68cb9d69092c1c (127.30.71.1). New cstate: current_term: 1 leader_uuid: "c2371e46865d4337bd68cb9d69092c1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } health_report { overall_health: HEALTHY } } }
I20260810 06:35:42.237285 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.234479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.30.71.1:40655)
0810 06:35:42.234561 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.234563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.234573 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:42.234644 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.234645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.234675 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.234752 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:42.234916 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.235456 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.236295 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.236507 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.236587 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.236848 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:42.237015 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:42.237105 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.237167 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.237190 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:35:42.239899 32722 raft_consensus.cc:1261] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Refusing update from remote peer c2371e46865d4337bd68cb9d69092c1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:42.240391 342 consensus_queue.cc:1035] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:42.241744 32543 raft_consensus.cc:1261] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Refusing update from remote peer c2371e46865d4337bd68cb9d69092c1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:42.242069 335 consensus_queue.cc:1035] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:42.244349 346 mvcc.cc:205] Tried to move back new op lower bound from 7316863968210554880 to 7316863968103673856. Current Snapshot: MvccSnapshot[applied={T|T < 7316863968210554880}]
I20260810 06:35:42.248898 32234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:42.249078 32234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:42.250274 32234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2: Sending DeleteTablet for 3 replicas of tablet 21a693610e9744f493d007042c9aaa2c
I20260810 06:35:42.250828 32541 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a693610e9744f493d007042c9aaa2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:42 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:41656
I20260810 06:35:42.250829 32390 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a693610e9744f493d007042c9aaa2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:42 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:45022
I20260810 06:35:42.251473 32234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:42.251654 32234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:42.251875 356 tablet_replica.cc:330] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: stopping tablet replica
I20260810 06:35:42.251989 356 raft_consensus.cc:2229] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:42.252295 32700 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a693610e9744f493d007042c9aaa2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:42 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:44890
I20260810 06:35:42.252445 356 raft_consensus.cc:2258] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:42.252616 357 tablet_replica.cc:330] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: stopping tablet replica
I20260810 06:35:42.252708 357 raft_consensus.cc:2229] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:42.253500 32390 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7f55b5909844808d397e55aac2b259 (DEFAULT_TABLE table=test-table [id=9a5a20ba91b54417b9295e599c210598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:42.253504 32700 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7f55b5909844808d397e55aac2b259 (DEFAULT_TABLE table=test-table [id=9a5a20ba91b54417b9295e599c210598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:42.253530 32541 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7f55b5909844808d397e55aac2b259 (DEFAULT_TABLE table=test-table [id=9a5a20ba91b54417b9295e599c210598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:42.253638 32541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7f55b5909844808d397e55aac2b259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.253639 32700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7f55b5909844808d397e55aac2b259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.253639 32390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7f55b5909844808d397e55aac2b259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.255177 329 tablet_bootstrap.cc:492] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: Bootstrap starting.
I20260810 06:35:42.255213 330 tablet_bootstrap.cc:492] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Bootstrap starting.
I20260810 06:35:42.255618 329 tablet_bootstrap.cc:654] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.255836 330 tablet_bootstrap.cc:654] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.256393 329 tablet_bootstrap.cc:492] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: No bootstrap required, opened a new log
I20260810 06:35:42.256429 356 ts_tablet_manager.cc:1905] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:42.256448 329 ts_tablet_manager.cc:1397] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:42.256580 329 raft_consensus.cc:348] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.256635 329 raft_consensus.cc:374] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.256649 329 raft_consensus.cc:729] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2371e46865d4337bd68cb9d69092c1c, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.256708 329 consensus_queue.cc:260] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.256803 329 ts_tablet_manager.cc:1428] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:42.257575 331 tablet_bootstrap.cc:492] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: Bootstrap starting.
I20260810 06:35:42.252803 357 raft_consensus.cc:2258] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:42.258191 331 tablet_bootstrap.cc:654] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.258424 358 tablet_replica.cc:330] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: stopping tablet replica
I20260810 06:35:42.258538 330 tablet_bootstrap.cc:492] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: No bootstrap required, opened a new log
I20260810 06:35:42.258595 330 ts_tablet_manager.cc:1397] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:35:42.258764 330 raft_consensus.cc:348] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.258843 330 raft_consensus.cc:374] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.258865 330 raft_consensus.cc:729] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d673bd4b5bd44e8c86e2b0a5b7b4ade2, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.258919 331 tablet_bootstrap.cc:492] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: No bootstrap required, opened a new log
I20260810 06:35:42.258908 330 consensus_queue.cc:260] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.258952 331 ts_tablet_manager.cc:1397] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:35:42.259016 330 ts_tablet_manager.cc:1428] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:42.258545 358 raft_consensus.cc:2229] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:42.259148 331 raft_consensus.cc:348] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.259402 331 raft_consensus.cc:374] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.259428 331 raft_consensus.cc:729] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edcffcd503a14f30a78a5289a92b5bc7, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.259434 358 raft_consensus.cc:2258] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:42.259472 331 consensus_queue.cc:260] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.259560 331 ts_tablet_manager.cc:1428] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:35:42.259827 357 ts_tablet_manager.cc:1905] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:42.260414 358 ts_tablet_manager.cc:1905] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:42.261667 356 ts_tablet_manager.cc:1918] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:35:42.261782 356 log.cc:1196] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/21a693610e9744f493d007042c9aaa2c
I20260810 06:35:42.261912 357 ts_tablet_manager.cc:1918] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:35:42.262014 358 ts_tablet_manager.cc:1918] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:35:42.262038 357 log.cc:1196] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/21a693610e9744f493d007042c9aaa2c
I20260810 06:35:42.262115 356 ts_tablet_manager.cc:1939] T 21a693610e9744f493d007042c9aaa2c P edcffcd503a14f30a78a5289a92b5bc7: Deleting consensus metadata
I20260810 06:35:42.262179 358 log.cc:1196] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/21a693610e9744f493d007042c9aaa2c
I20260810 06:35:42.262322 357 ts_tablet_manager.cc:1939] T 21a693610e9744f493d007042c9aaa2c P c2371e46865d4337bd68cb9d69092c1c: Deleting consensus metadata
I20260810 06:35:42.262538 358 ts_tablet_manager.cc:1939] T 21a693610e9744f493d007042c9aaa2c P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Deleting consensus metadata
I20260810 06:35:42.262751 32220 catalog_manager.cc:4841] TS c2371e46865d4337bd68cb9d69092c1c (127.30.71.1:40655): tablet 21a693610e9744f493d007042c9aaa2c (table test-table [id=80bd672141da4491a3f4ac0d0f664d90]) successfully deleted
I20260810 06:35:42.262953 32220 catalog_manager.cc:4841] TS d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345): tablet 21a693610e9744f493d007042c9aaa2c (table test-table [id=80bd672141da4491a3f4ac0d0f664d90]) successfully deleted
I20260810 06:35:42.262751 32219 catalog_manager.cc:4841] TS edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2:34347): tablet 21a693610e9744f493d007042c9aaa2c (table test-table [id=80bd672141da4491a3f4ac0d0f664d90]) successfully deleted
W20260810 06:35:42.298882 307 tablet.cc:2369] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:35:42.412380 335 raft_consensus.cc:482] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:42.412515 335 raft_consensus.cc:504] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.412755 335 leader_election.cc:288] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2:34347), d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345)
I20260810 06:35:42.413057 32543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7f55b5909844808d397e55aac2b259" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edcffcd503a14f30a78a5289a92b5bc7" is_pre_election: true
I20260810 06:35:42.413057 32722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7f55b5909844808d397e55aac2b259" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" is_pre_election: true
I20260810 06:35:42.413174 32722 raft_consensus.cc:2454] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 0.
I20260810 06:35:42.413174 32543 raft_consensus.cc:2454] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 0.
I20260810 06:35:42.413360 32338 leader_election.cc:302] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2371e46865d4337bd68cb9d69092c1c, d673bd4b5bd44e8c86e2b0a5b7b4ade2; no voters:
I20260810 06:35:42.413501 335 raft_consensus.cc:2781] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:42.413583 335 raft_consensus.cc:482] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:42.413617 335 raft_consensus.cc:3037] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.414371 335 raft_consensus.cc:504] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.414527 335 leader_election.cc:288] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 election: Requested vote from peers edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2:34347), d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345)
I20260810 06:35:42.414718 32543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7f55b5909844808d397e55aac2b259" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edcffcd503a14f30a78a5289a92b5bc7"
I20260810 06:35:42.414719 32722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7f55b5909844808d397e55aac2b259" candidate_uuid: "c2371e46865d4337bd68cb9d69092c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2"
I20260810 06:35:42.414799 32543 raft_consensus.cc:3037] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.414819 32722 raft_consensus.cc:3037] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.415508 32722 raft_consensus.cc:2454] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 1.
I20260810 06:35:42.415519 32543 raft_consensus.cc:2454] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2371e46865d4337bd68cb9d69092c1c in term 1.
I20260810 06:35:42.415684 32336 leader_election.cc:302] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2371e46865d4337bd68cb9d69092c1c, edcffcd503a14f30a78a5289a92b5bc7; no voters:
I20260810 06:35:42.415799 335 raft_consensus.cc:2781] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:42.415867 335 raft_consensus.cc:686] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 1 LEADER]: Becoming Leader. State: Replica: c2371e46865d4337bd68cb9d69092c1c, State: Running, Role: LEADER
I20260810 06:35:42.415923 335 consensus_queue.cc:237] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.416605 32233 catalog_manager.cc:5495] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c reported cstate change: term changed from 0 to 1, leader changed from <none> to c2371e46865d4337bd68cb9d69092c1c (127.30.71.1). New cstate: current_term: 1 leader_uuid: "c2371e46865d4337bd68cb9d69092c1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:42.491293 368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.483806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35355 (local address 127.0.0.1:54488)
0810 06:35:42.483969 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.483982 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.484057 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.484193 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.484194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.484507 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:42.484557 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.484559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.485307 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.485308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.485892 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.485894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.486231 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.486232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.486272 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.486905 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:42.486910 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:42.488373 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:42.490105 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:42.490113 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:42.490115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:42.490319 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:42.490523 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:42.490524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:42.490524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:42.490601 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:42.490875 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:42.490883 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:42.490930 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.491080 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.491183 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260810 06:35:42.491336 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.483866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54488 (local address 127.30.71.62:35355)
0810 06:35:42.483943 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.483945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.484056 (+ 111us) server_negotiation.cc:407] Connection header received
0810 06:35:42.484102 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.484103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.484112 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.484176 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:42.484678 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.485229 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.485965 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.486151 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.486218 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.488458 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:42.488459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:42.488460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:42.488466 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:42.490027 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.490384 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.490384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.490385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.490437 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.490678 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.490678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.490678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.490797 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:42.490846 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.491115 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.491233 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":41}
I20260810 06:35:42.493687 32233 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:54488:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:35:42.493999 32233 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:42.495568 32700 tablet_service.cc:1467] Processing CreateTablet for tablet 267f4db5de4c4738bb8cabe98cf4b5f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd818ba4b8574333803f3d5132d451c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:35:42.495599 32541 tablet_service.cc:1467] Processing CreateTablet for tablet 267f4db5de4c4738bb8cabe98cf4b5f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd818ba4b8574333803f3d5132d451c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:35:42.495795 32700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267f4db5de4c4738bb8cabe98cf4b5f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.495836 32541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267f4db5de4c4738bb8cabe98cf4b5f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.496935 330 tablet_bootstrap.cc:492] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Bootstrap starting.
I20260810 06:35:42.497236 331 tablet_bootstrap.cc:492] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: Bootstrap starting.
I20260810 06:35:42.497591 330 tablet_bootstrap.cc:654] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.497701 331 tablet_bootstrap.cc:654] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.495565 32390 tablet_service.cc:1467] Processing CreateTablet for tablet 267f4db5de4c4738bb8cabe98cf4b5f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd818ba4b8574333803f3d5132d451c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:35:42.497926 32390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267f4db5de4c4738bb8cabe98cf4b5f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:42.498448 331 tablet_bootstrap.cc:492] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: No bootstrap required, opened a new log
I20260810 06:35:42.498489 331 ts_tablet_manager.cc:1397] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:42.498607 331 raft_consensus.cc:348] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.498662 331 raft_consensus.cc:374] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.498677 331 raft_consensus.cc:729] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edcffcd503a14f30a78a5289a92b5bc7, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.498745 331 consensus_queue.cc:260] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.498848 331 ts_tablet_manager.cc:1428] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:42.498917 336 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.499023 336 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:42.499266 330 tablet_bootstrap.cc:492] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: No bootstrap required, opened a new log
I20260810 06:35:42.499339 330 ts_tablet_manager.cc:1397] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:35:42.499701 330 raft_consensus.cc:348] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.499809 330 raft_consensus.cc:374] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.499833 330 raft_consensus.cc:729] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d673bd4b5bd44e8c86e2b0a5b7b4ade2, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.499876 330 consensus_queue.cc:260] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.499987 330 ts_tablet_manager.cc:1428] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 06:35:42.500128 337 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.500203 329 tablet_bootstrap.cc:492] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: Bootstrap starting.
I20260810 06:35:42.500253 337 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:42.500550 329 tablet_bootstrap.cc:654] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:42.501274 329 tablet_bootstrap.cc:492] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: No bootstrap required, opened a new log
I20260810 06:35:42.501341 329 ts_tablet_manager.cc:1397] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:42.501454 329 raft_consensus.cc:348] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.501536 329 raft_consensus.cc:374] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:42.501564 329 raft_consensus.cc:729] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2371e46865d4337bd68cb9d69092c1c, State: Initialized, Role: FOLLOWER
I20260810 06:35:42.501612 329 consensus_queue.cc:260] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.501708 329 ts_tablet_manager.cc:1428] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:42.501739 335 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.501825 335 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:42.553095 336 raft_consensus.cc:482] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:42.553202 336 raft_consensus.cc:504] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.553642 336 leader_election.cc:288] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2371e46865d4337bd68cb9d69092c1c (127.30.71.1:40655), d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345)
I20260810 06:35:42.557534 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.553995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:37849 (local address 127.30.71.1:40655)
0810 06:35:42.554106 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.554109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.554288 (+ 179us) server_negotiation.cc:407] Connection header received
0810 06:35:42.554369 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.554371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.554382 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.554441 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:42.554782 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.555320 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.556774 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.557121 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.557176 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.557185 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.557401 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.557456 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:35:42.557548 372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.553898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:40655 (local address 127.30.71.2:37849)
0810 06:35:42.554118 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.554131 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.554292 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.554466 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.554467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.554469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:42.554692 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.554695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.555410 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.555411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.556659 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.556663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.557241 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.557243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.557317 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.557321 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.557385 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.557418 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":34,"thread_start_us":109,"threads_started":1}
I20260810 06:35:42.557683 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.554324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:39837 (local address 127.30.71.3:45345)
0810 06:35:42.554389 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.554391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.554402 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:35:42.554477 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.554478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.554488 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.554554 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:42.554691 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.555341 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.556724 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.557184 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.557276 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.557286 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.557524 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.557581 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:35:42.557894 32411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f4db5de4c4738bb8cabe98cf4b5f5" candidate_uuid: "edcffcd503a14f30a78a5289a92b5bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2371e46865d4337bd68cb9d69092c1c" is_pre_election: true
I20260810 06:35:42.558027 32411 raft_consensus.cc:2454] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edcffcd503a14f30a78a5289a92b5bc7 in term 0.
I20260810 06:35:42.558081 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.554220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:45345 (local address 127.30.71.2:39837)
0810 06:35:42.554340 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.554350 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.554409 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.554559 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.554559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.554561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:42.554616 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.554617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.555448 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.555449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.556641 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.556644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.557272 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.557273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.557433 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.557438 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.557498 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.557538 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":40,"thread_start_us":64,"threads_started":1}
I20260810 06:35:42.558326 32491 leader_election.cc:302] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2371e46865d4337bd68cb9d69092c1c, edcffcd503a14f30a78a5289a92b5bc7; no voters:
I20260810 06:35:42.558383 32722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f4db5de4c4738bb8cabe98cf4b5f5" candidate_uuid: "edcffcd503a14f30a78a5289a92b5bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" is_pre_election: true
I20260810 06:35:42.558471 32722 raft_consensus.cc:2454] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edcffcd503a14f30a78a5289a92b5bc7 in term 0.
I20260810 06:35:42.558557 336 raft_consensus.cc:2781] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:42.558631 336 raft_consensus.cc:482] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:42.558656 336 raft_consensus.cc:3037] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.559490 336 raft_consensus.cc:504] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.559623 336 leader_election.cc:288] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [CANDIDATE]: Term 1 election: Requested vote from peers c2371e46865d4337bd68cb9d69092c1c (127.30.71.1:40655), d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345)
I20260810 06:35:42.559820 32722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f4db5de4c4738bb8cabe98cf4b5f5" candidate_uuid: "edcffcd503a14f30a78a5289a92b5bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2"
I20260810 06:35:42.559937 32722 raft_consensus.cc:3037] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.559909 32411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f4db5de4c4738bb8cabe98cf4b5f5" candidate_uuid: "edcffcd503a14f30a78a5289a92b5bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2371e46865d4337bd68cb9d69092c1c"
I20260810 06:35:42.559981 32411 raft_consensus.cc:3037] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:42.560850 32411 raft_consensus.cc:2454] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edcffcd503a14f30a78a5289a92b5bc7 in term 1.
I20260810 06:35:42.560859 32722 raft_consensus.cc:2454] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edcffcd503a14f30a78a5289a92b5bc7 in term 1.
I20260810 06:35:42.561059 32491 leader_election.cc:302] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d673bd4b5bd44e8c86e2b0a5b7b4ade2, edcffcd503a14f30a78a5289a92b5bc7; no voters:
I20260810 06:35:42.561190 336 raft_consensus.cc:2781] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:42.561383 336 raft_consensus.cc:686] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [term 1 LEADER]: Becoming Leader. State: Replica: edcffcd503a14f30a78a5289a92b5bc7, State: Running, Role: LEADER
I20260810 06:35:42.561446 336 consensus_queue.cc:237] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } }
I20260810 06:35:42.561851 374 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: TxnStatusTablet state changed. Reason: New leader edcffcd503a14f30a78a5289a92b5bc7. Latest consensus state: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.561956 374 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:35:42.562148 376 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:35:42.562276 376 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:35:42.562296 32233 catalog_manager.cc:5495] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } health_report { overall_health: UNKNOWN } } }
I20260810 06:35:42.577090 325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.574097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41668 (local address 127.30.71.2:34347)
0810 06:35:42.574168 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.574169 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.574180 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:35:42.574293 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.574294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.574310 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.574367 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:42.574496 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.575042 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.576267 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.576451 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.576514 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.576740 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:42.576838 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:42.576894 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.576982 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.577002 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:35:42.577126 368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.574029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:34347 (local address 127.0.0.1:41668)
0810 06:35:42.574127 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.574141 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.574202 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.574372 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.574373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.574374 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:42.574432 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.574435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.575103 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.575104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.576151 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.576154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.576535 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.576536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.576610 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.576630 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:42.576906 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:42.576915 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.576978 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.577032 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:35:42.579057 32722 raft_consensus.cc:1261] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Refusing update from remote peer edcffcd503a14f30a78a5289a92b5bc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:42.579069 32411 raft_consensus.cc:1261] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Refusing update from remote peer edcffcd503a14f30a78a5289a92b5bc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:42.579356 374 consensus_queue.cc:1035] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:42.579496 336 consensus_queue.cc:1035] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:42.580128 335 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: TxnStatusTablet state changed. Reason: New leader edcffcd503a14f30a78a5289a92b5bc7. Latest consensus state: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.580132 337 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: TxnStatusTablet state changed. Reason: New leader edcffcd503a14f30a78a5289a92b5bc7. Latest consensus state: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.580229 335 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:42.580236 337 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:42.580736 349 mvcc.cc:205] Tried to move back new op lower bound from 7316863969601220608 to 7316863969532338176. Current Snapshot: MvccSnapshot[applied={T|T < 7316863969601220608}]
I20260810 06:35:42.580969 336 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.581060 336 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:35:42.581610 374 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.581693 374 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P edcffcd503a14f30a78a5289a92b5bc7: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:35:42.581913 335 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.581991 335 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P c2371e46865d4337bd68cb9d69092c1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:42.582067 347 mvcc.cc:205] Tried to move back new op lower bound from 7316863969601220608 to 7316863969532338176. Current Snapshot: MvccSnapshot[applied={T|T < 7316863969601220608}]
I20260810 06:35:42.582425 337 tablet_replica.cc:439] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "edcffcd503a14f30a78a5289a92b5bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2371e46865d4337bd68cb9d69092c1c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40655 } } peers { permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 } } peers { permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 } } }
I20260810 06:35:42.582511 337 tablet_replica.cc:442] T 267f4db5de4c4738bb8cabe98cf4b5f5 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:42.588284 325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.585783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:45623 (local address 127.30.71.2:34347)
0810 06:35:42.585851 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.585853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.585915 (+ 62us) server_negotiation.cc:407] Connection header received
0810 06:35:42.585990 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.585991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.586001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.586054 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:42.586257 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.586695 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.587592 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.587779 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.587877 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.588041 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:42.588079 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:42.588142 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.588198 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.588216 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:35:42.588320 385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.585679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:34347 (local address 127.30.71.1:45623)
0810 06:35:42.585859 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.585871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.585932 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.586058 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.586058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.586060 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:42.586120 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.586121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.586786 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.586787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.587502 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.587504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.587867 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.587868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.587934 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.587949 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:42.588144 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:42.588146 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.588193 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.588240 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260810 06:35:42.590938 32722 raft_consensus.cc:1261] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Refusing update from remote peer c2371e46865d4337bd68cb9d69092c1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:42.590940 32543 raft_consensus.cc:1261] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Refusing update from remote peer c2371e46865d4337bd68cb9d69092c1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:42.591248 342 consensus_queue.cc:1035] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d673bd4b5bd44e8c86e2b0a5b7b4ade2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:42.591351 335 consensus_queue.cc:1035] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "edcffcd503a14f30a78a5289a92b5bc7" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:42.600703 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.597944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:41011 (local address 127.30.71.1:40655)
0810 06:35:42.597982 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.597985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.598125 (+ 140us) server_negotiation.cc:407] Connection header received
0810 06:35:42.598197 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.598198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.598210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.598266 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:42.598424 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.598884 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.600011 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.600233 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.600306 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.600476 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:42.600510 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:42.600553 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.600608 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.600630 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:35:42.600754 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.597888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:40655 (local address 127.30.71.2:41011)
0810 06:35:42.598050 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:42.598068 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:42.598140 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:42.598268 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:42.598269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:42.598283 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:42.598341 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.598342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.598968 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.598969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.599925 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:42.599928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.600305 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.600306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:42.600371 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.600394 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:42.600565 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:42.600566 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:35:42.600608 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 06:35:42.600648 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260810 06:35:42.609225 32235 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:42.609372 32235 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:42.610108 32235 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e3dd7322ad7475597d4090b78c928c2: Sending DeleteTablet for 3 replicas of tablet 3e7f55b5909844808d397e55aac2b259
I20260810 06:35:42.610505 32390 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7f55b5909844808d397e55aac2b259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:42 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:45022
I20260810 06:35:42.610628 357 tablet_replica.cc:330] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: stopping tablet replica
I20260810 06:35:42.610639 32700 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7f55b5909844808d397e55aac2b259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:42 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:44890
I20260810 06:35:42.610714 357 raft_consensus.cc:2229] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:42.610781 357 raft_consensus.cc:2258] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:42.610783 358 tablet_replica.cc:330] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: stopping tablet replica
I20260810 06:35:42.610841 358 raft_consensus.cc:2229] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:42.610879 358 raft_consensus.cc:2258] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:42.611137 357 ts_tablet_manager.cc:1905] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:42.611316 358 ts_tablet_manager.cc:1905] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:42.612015 32541 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7f55b5909844808d397e55aac2b259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:42 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:41656
I20260810 06:35:42.612129 356 tablet_replica.cc:330] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: stopping tablet replica
I20260810 06:35:42.612172 356 raft_consensus.cc:2229] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:42.612223 356 raft_consensus.cc:2258] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:42.612601 356 ts_tablet_manager.cc:1905] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:42.613479 358 ts_tablet_manager.cc:1918] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:42.613497 357 ts_tablet_manager.cc:1918] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:42.613577 357 log.cc:1196] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/3e7f55b5909844808d397e55aac2b259
I20260810 06:35:42.613579 358 log.cc:1196] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/3e7f55b5909844808d397e55aac2b259
I20260810 06:35:42.613871 357 ts_tablet_manager.cc:1939] T 3e7f55b5909844808d397e55aac2b259 P c2371e46865d4337bd68cb9d69092c1c: Deleting consensus metadata
I20260810 06:35:42.613933 358 ts_tablet_manager.cc:1939] T 3e7f55b5909844808d397e55aac2b259 P d673bd4b5bd44e8c86e2b0a5b7b4ade2: Deleting consensus metadata
I20260810 06:35:42.614270 32220 catalog_manager.cc:4841] TS c2371e46865d4337bd68cb9d69092c1c (127.30.71.1:40655): tablet 3e7f55b5909844808d397e55aac2b259 (table test-table [id=9a5a20ba91b54417b9295e599c210598]) successfully deleted
I20260810 06:35:42.614370 32220 catalog_manager.cc:4841] TS d673bd4b5bd44e8c86e2b0a5b7b4ade2 (127.30.71.3:45345): tablet 3e7f55b5909844808d397e55aac2b259 (table test-table [id=9a5a20ba91b54417b9295e599c210598]) successfully deleted
I20260810 06:35:42.614423 356 ts_tablet_manager.cc:1918] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:42.614492 356 log.cc:1196] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/3e7f55b5909844808d397e55aac2b259
I20260810 06:35:42.614764 356 ts_tablet_manager.cc:1939] T 3e7f55b5909844808d397e55aac2b259 P edcffcd503a14f30a78a5289a92b5bc7: Deleting consensus metadata
I20260810 06:35:42.615203 32219 catalog_manager.cc:4841] TS edcffcd503a14f30a78a5289a92b5bc7 (127.30.71.2:34347): tablet 3e7f55b5909844808d397e55aac2b259 (table test-table [id=9a5a20ba91b54417b9295e599c210598]) successfully deleted
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:35:42.623237 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.613074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.30.71.1:40655)
0810 06:35:42.613125 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.613128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.613192 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:35:42.613280 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.613281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.613296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.613359 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:42.615051 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.615672 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.616441 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.616661 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.616756 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.619967 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:42.619979 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:42.619979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:42.619988 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:42.621836 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.622193 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.622197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.622197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.622267 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.622522 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.622524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.622524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.622773 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:42.622956 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.623030 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.623131 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 06:35:42.623611 32450 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45036
I20260810 06:35:42.630851 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:42.624734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.30.71.62:35355)
0810 06:35:42.624804 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:42.624807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:42.624815 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:35:42.624843 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:42.624844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:42.624854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:42.624900 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:42.625411 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.625908 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.626596 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:42.626826 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:42.626902 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:42.628157 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:42.628158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:42.628158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:42.628167 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:42.629593 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.630010 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.630011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.630012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.630074 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:42.630285 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:42.630286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:42.630287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:42.630422 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:42.630533 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:42.630657 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 06:35:42.630744 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 06:35:42.631122 32235 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54490
I20260810 06:35:42.631827 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 32315
I20260810 06:35:42.643563 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 32470
I20260810 06:35:42.650590 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 32624
I20260810 06:35:42.657541 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 32199
2026-08-10T06:35:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3912 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[399](info): set up 2 sockets
Aug 10 06:35:42 dist-test-slave-w206 krb5kdc[399](info): commencing operation
krb5kdc: starting...
W20260810 06:35:43.758864 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:43 dist-test-slave-w206 krb5kdc[399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:43.884277 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42777
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:43.967276 413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:43.967492 413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:43.967514 413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:43.968617 413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:43.968652 413 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:35:43.968665 413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:43.968677 413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:43.968689 413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:43.968701 413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:43.970110 413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:42777
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:43.970322 413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:43.970490 413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:43.975513 418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:43.975508 419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:43.975507 421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:43.975772 413 server_base.cc:1034] running on GCE node
I20260810 06:35:43.976022 413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:43.976325 413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:43.977504 413 hybrid_clock.cc:648] HybridClock initialized: now 1786343743977489 us; error 45 us; skew 500 ppm
Aug 10 06:35:43 dist-test-slave-w206 krb5kdc[399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343743, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:43.980234 413 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:43.980742 413 webserver.cc:460] Webserver started at http://127.30.71.62:43639/ using document root <none> and password file <none>
I20260810 06:35:43.980934 413 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:43.980978 413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:43.981067 413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:43.982206 413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "9fa47fb4da4e42f2b17c435bb27416b3"
format_stamp: "Formatted at 2026-08-10 06:35:43 on dist-test-slave-w206"
server_key: "bac9c6437ab688820937ba7b4e1febfd"
server_key_iv: "6b680620ab0a6146051dfbfc3b53e96a"
server_key_version: "encryptionkey@0"
I20260810 06:35:43.982512 413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "9fa47fb4da4e42f2b17c435bb27416b3"
format_stamp: "Formatted at 2026-08-10 06:35:43 on dist-test-slave-w206"
server_key: "bac9c6437ab688820937ba7b4e1febfd"
server_key_iv: "6b680620ab0a6146051dfbfc3b53e96a"
server_key_version: "encryptionkey@0"
I20260810 06:35:43.983956 413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:43.984742 428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:43.985064 413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:43.985149 413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "9fa47fb4da4e42f2b17c435bb27416b3"
format_stamp: "Formatted at 2026-08-10 06:35:43 on dist-test-slave-w206"
server_key: "bac9c6437ab688820937ba7b4e1febfd"
server_key_iv: "6b680620ab0a6146051dfbfc3b53e96a"
server_key_version: "encryptionkey@0"
I20260810 06:35:43.985268 413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:43.998708 413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:44.001080 413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:44.001245 413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:44.006126 413 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:42777
I20260810 06:35:44.006206 496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:42777 every 8 connection(s)
I20260810 06:35:44.006563 413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:44.007371 497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.009727 497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Bootstrap starting.
I20260810 06:35:44.010372 497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.010377 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 413
I20260810 06:35:44.010479 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:44.010663 31004 external_mini_cluster.cc:1428] Setting key 90e3ec69509ca2a8231d90516435c1d7
I20260810 06:35:44.010710 497 log.cc:824] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:44.011636 497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: No bootstrap required, opened a new log
I20260810 06:35:44.013700 497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42777 } }
I20260810 06:35:44.013844 497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.013867 497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fa47fb4da4e42f2b17c435bb27416b3, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.014005 497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42777 } }
I20260810 06:35:44.014093 497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:44.014137 497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:44.014184 497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:44.014810 497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42777 } }
I20260810 06:35:44.014928 497 leader_election.cc:302] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fa47fb4da4e42f2b17c435bb27416b3; no voters:
I20260810 06:35:44.015126 497 leader_election.cc:288] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:44.015236 502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:44.015383 497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:44.015395 502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [term 1 LEADER]: Becoming Leader. State: Replica: 9fa47fb4da4e42f2b17c435bb27416b3, State: Running, Role: LEADER
I20260810 06:35:44.015604 502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42777 } }
I20260810 06:35:44.016907 503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fa47fb4da4e42f2b17c435bb27416b3. Latest consensus state: current_term: 1 leader_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42777 } } }
I20260810 06:35:44.017143 503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:44.017320 504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa47fb4da4e42f2b17c435bb27416b3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42777 } } }
I20260810 06:35:44.017385 504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3 [sys.catalog]: This master's current role is: LEADER
W20260810 06:35:44.017493 516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:35:44.017556 516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:35:44.017637 517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:44.017876 517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:44.019872 517 catalog_manager.cc:1347] Generated new cluster ID: d9a0a947736541cc906a2811ff171e95
I20260810 06:35:44.019969 517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:44.023085 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.011280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.30.71.62:42777)
0810 06:35:44.011466 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.011475 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.011511 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:35:44.012012 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.012019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.012076 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.012240 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.013005 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.014612 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.015714 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.016196 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.016280 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.019203 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.019207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.019207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.019219 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.021347 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.021753 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.021757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.021757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.021870 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.022076 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.022077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.022077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.022390 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.022559 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.022632 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.022790 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260810 06:35:44.024523 517 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:44.025033 517 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:44.028337 517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Generated new TSK 0
I20260810 06:35:44.028502 517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:44.065878 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42777
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:44.150012 524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:44.150226 524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:44.150264 524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:44.151582 524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:44.151629 524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:44.151651 524 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:35:44.151664 524 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:35:44.151726 524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:44.153245 524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:44.153517 524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:44.153793 524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:44.160490 533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:44.160487 531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:44.160482 530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:44.160971 524 server_base.cc:1034] running on GCE node
I20260810 06:35:44.161209 524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:44.161490 524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:44.162671 524 hybrid_clock.cc:648] HybridClock initialized: now 1786343744162668 us; error 42 us; skew 500 ppm
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:44.165643 524 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:44.166234 524 webserver.cc:460] Webserver started at http://127.30.71.1:43799/ using document root <none> and password file <none>
I20260810 06:35:44.166509 524 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:44.166584 524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:44.166764 524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:44.168174 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "77b530c78fb948d696a93bbb2f8afcdc"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "3d6a5fd875fb76cee05b97cb3cd9cde4"
server_key_iv: "79d3014eb1626e37d6abc687c8a76071"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.168668 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "77b530c78fb948d696a93bbb2f8afcdc"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "3d6a5fd875fb76cee05b97cb3cd9cde4"
server_key_iv: "79d3014eb1626e37d6abc687c8a76071"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.170239 524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:35:44.171185 540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.171402 524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:44.171510 524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "77b530c78fb948d696a93bbb2f8afcdc"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "3d6a5fd875fb76cee05b97cb3cd9cde4"
server_key_iv: "79d3014eb1626e37d6abc687c8a76071"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.171598 524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:44.182665 524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:44.185482 524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:44.185654 524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:44.186267 524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:44.186329 524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.186383 524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:44.186411 524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:44.194088 524 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:44041
I20260810 06:35:44.194406 664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:44041 every 8 connection(s)
I20260810 06:35:44.194559 524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:44.196676 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.186794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:58117 (local address 127.30.71.62:42777)
0810 06:35:44.186841 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.186843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.187146 (+ 303us) server_negotiation.cc:407] Connection header received
0810 06:35:44.187728 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.187730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.187745 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.187798 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.188603 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.189415 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.190490 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.190755 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.190848 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.193801 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.193803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.193804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.193814 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.195468 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.195925 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.195926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.195926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.195981 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.196196 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.196196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.196196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.196294 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.196339 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.196539 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.196612 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:35:44.196952 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.186756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42777 (local address 127.30.71.1:58117)
0810 06:35:44.186966 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.187050 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.187593 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.187894 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.187899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.188245 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:44.188496 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.188501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.189523 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.189524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.190390 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.190392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.190864 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.190865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.191042 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.191671 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:44.191681 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:44.193677 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:44.195549 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.195555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.195556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.195853 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.196044 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.196045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.196045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.196146 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.196330 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:44.196332 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:44.196448 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.196527 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.196664 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260810 06:35:44.197484 665 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42777
I20260810 06:35:44.197570 665 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:44.197765 665 heartbeater.cc:511] Master 127.30.71.62:42777 requested a full tablet report, sending...
I20260810 06:35:44.198348 448 ts_manager.cc:194] Registered new tserver with Master: 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041)
I20260810 06:35:44.199126 448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:58117
I20260810 06:35:44.201653 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 524
I20260810 06:35:44.201761 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:44.202005 31004 external_mini_cluster.cc:1428] Setting key 174075f25fd15ce4ca71bde116f3e7ce
I20260810 06:35:44.204744 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.198701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:41665 (local address 127.30.71.62:42777)
0810 06:35:44.199063 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.199064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.199071 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:35:44.199087 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.199087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.199095 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.199159 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.199574 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.200038 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.200735 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.200908 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.200957 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.201942 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.201943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.201944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.201951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.203498 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.203930 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.203931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.203931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.203980 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.204191 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.204192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.204192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.204285 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.204357 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.204525 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.204613 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260810 06:35:44.204756 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.198636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42777 (local address 127.30.71.1:41665)
0810 06:35:44.198770 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.198782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.198835 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.199183 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.199184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.199396 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:44.199493 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.199496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.200135 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.200136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.200662 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.200663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.200973 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.200974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.201030 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.201498 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:44.201503 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:44.201860 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:44.203579 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.203580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.203581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.203831 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.204052 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.204053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.204054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.204131 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.204348 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:44.204349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:44.204403 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.204477 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.204605 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:44.230695 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42777
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:44.320346 675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:44.320652 675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:44.320698 675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:44.322911 675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:44.322995 675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:44.323032 675 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:35:44.323062 675 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:35:44.323087 675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:44.325009 675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:44.325286 675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:44.325726 675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:44.331336 680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:44.331351 681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:44.331336 683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:44.331620 675 server_base.cc:1034] running on GCE node
I20260810 06:35:44.331841 675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:44.332086 675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:44.333256 675 hybrid_clock.cc:648] HybridClock initialized: now 1786343744333243 us; error 29 us; skew 500 ppm
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:44.336083 675 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:44.336586 675 webserver.cc:460] Webserver started at http://127.30.71.2:44853/ using document root <none> and password file <none>
I20260810 06:35:44.336835 675 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:44.336905 675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:44.337059 675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:44.338160 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "47008aebce6a49ecbc21204b7f13903d"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "fb0ae8057f2d9019b9feb8cf9fe2e5aa"
server_key_iv: "6f223e700df2888442cacdca31ff02d4"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.338526 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "47008aebce6a49ecbc21204b7f13903d"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "fb0ae8057f2d9019b9feb8cf9fe2e5aa"
server_key_iv: "6f223e700df2888442cacdca31ff02d4"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.339993 675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:35:44.340778 690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.340955 675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:44.341063 675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "47008aebce6a49ecbc21204b7f13903d"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "fb0ae8057f2d9019b9feb8cf9fe2e5aa"
server_key_iv: "6f223e700df2888442cacdca31ff02d4"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.341140 675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:44.357301 675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:44.359500 675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:44.359663 675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:44.360277 675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:44.360338 675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.360383 675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:44.360432 675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:44.368203 675 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:43919
I20260810 06:35:44.368333 814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:43919 every 8 connection(s)
I20260810 06:35:44.368685 675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:35:44.370850 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.360768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:37891 (local address 127.30.71.62:42777)
0810 06:35:44.360846 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.360848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.361095 (+ 247us) server_negotiation.cc:407] Connection header received
0810 06:35:44.361618 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.361620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.361634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.361711 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.362699 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.363349 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.364473 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.364796 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.364897 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.367894 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.367896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.367897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.367905 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.369609 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.370003 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.370004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.370004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.370059 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.370305 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.370306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.370306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.370420 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.370472 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.370711 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.370784 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:35:44.371100 699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.360720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42777 (local address 127.30.71.2:37891)
0810 06:35:44.360931 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.360980 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.361509 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.361831 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.361834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.362223 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:44.362563 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.362569 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.363455 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.363456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.364320 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.364323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.364909 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.364910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.365051 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.365616 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:44.365631 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:44.367778 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:44.369685 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.369690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.369691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.369928 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.370118 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.370118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.370119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.370219 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.370473 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:44.370478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:44.370597 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.370680 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.370813 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":82,"threads_started":1}
I20260810 06:35:44.371201 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 675
I20260810 06:35:44.371321 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:44.371546 31004 external_mini_cluster.cc:1428] Setting key d120c22f5507ba3393d492e5b5c8cf80
I20260810 06:35:44.371698 815 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42777
I20260810 06:35:44.371802 815 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:44.372184 815 heartbeater.cc:511] Master 127.30.71.62:42777 requested a full tablet report, sending...
I20260810 06:35:44.372608 447 ts_manager.cc:194] Registered new tserver with Master: 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919)
I20260810 06:35:44.373014 447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:37891
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
I20260810 06:35:44.381348 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.374857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:49363 (local address 127.30.71.62:42777)
0810 06:35:44.374904 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.374907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.375075 (+ 168us) server_negotiation.cc:407] Connection header received
0810 06:35:44.375150 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.375151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.375162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.375218 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.375571 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.376053 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.376839 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.377153 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.377232 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.378202 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.378204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.378204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.378212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.379855 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.380245 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.380246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.380247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.380314 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.380645 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.380646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.380646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.380864 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.380940 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.381120 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.381216 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:35:44.381373 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.374794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42777 (local address 127.30.71.2:49363)
0810 06:35:44.375013 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.375027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.375091 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.375223 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.375224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.375421 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:44.375481 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.375483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.376149 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.376150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.376752 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.376753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.377234 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.377235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.377301 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.377800 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:44.377805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:44.378111 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:44.379967 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.379967 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.379968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.380152 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.380417 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.380418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.380418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.380556 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.380948 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:44.380949 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:44.380995 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.381070 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.381258 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:44.401717 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42777
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:44.484790 825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:44.485004 825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:44.485029 825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:44.486145 825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:44.486179 825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:44.486199 825 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:35:44.486212 825 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:35:44.486228 825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:44.487660 825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:44.487921 825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:44.488165 825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:44.494016 834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:44.494079 825 server_base.cc:1034] running on GCE node
W20260810 06:35:44.494064 832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:44.494064 831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:44.494505 825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:44.494728 825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:44.495903 825 hybrid_clock.cc:648] HybridClock initialized: now 1786343744495881 us; error 40 us; skew 500 ppm
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:44.499090 825 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:44.499660 825 webserver.cc:460] Webserver started at http://127.30.71.3:33833/ using document root <none> and password file <none>
I20260810 06:35:44.499953 825 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:44.500005 825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:44.500195 825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:44.501402 825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "224257b2965242a58807c242ef04a1cc"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "106b5fb5dda119b3404b19016d46f3fd"
server_key_iv: "48b1599930788fa2bcd906ce56d8a859"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.501794 825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "224257b2965242a58807c242ef04a1cc"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "106b5fb5dda119b3404b19016d46f3fd"
server_key_iv: "48b1599930788fa2bcd906ce56d8a859"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.503314 825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:44.504184 841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.504407 825 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:44.504508 825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "224257b2965242a58807c242ef04a1cc"
format_stamp: "Formatted at 2026-08-10 06:35:44 on dist-test-slave-w206"
server_key: "106b5fb5dda119b3404b19016d46f3fd"
server_key_iv: "48b1599930788fa2bcd906ce56d8a859"
server_key_version: "encryptionkey@0"
I20260810 06:35:44.504588 825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:44.533090 825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:44.535863 825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:44.536054 825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:44.536638 825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:44.536685 825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.536724 825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:44.536777 825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:44.546252 825 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:46455
I20260810 06:35:44.546294 962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:46455 every 8 connection(s)
I20260810 06:35:44.546662 825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:35:44.547780 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 825
I20260810 06:35:44.547890 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:44.548091 31004 external_mini_cluster.cc:1428] Setting key 3a41759ff78b33996a61332b476cd9d7
I20260810 06:35:44.548679 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.537255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:42039 (local address 127.30.71.62:42777)
0810 06:35:44.537314 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.537316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.537675 (+ 359us) server_negotiation.cc:407] Connection header received
0810 06:35:44.538143 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.538145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.538161 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.538239 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.539224 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.540205 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.541820 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.542225 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.542319 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.545116 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.545117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.545118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.545128 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.546954 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.547491 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.547493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.547493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.547581 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.547867 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.547869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.547869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.548036 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.548122 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.548484 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.548583 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:35:44.548867 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.537255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42777 (local address 127.30.71.3:42039)
0810 06:35:44.537521 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.537573 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.538046 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.538452 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.538457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.538857 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:44.539139 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.539145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.540348 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.540349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.541700 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.541704 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.542342 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.542343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.542493 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.543168 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:44.543177 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:44.545007 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:44.547084 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.547091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.547093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.547405 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.547676 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.547677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.547677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.547787 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.548112 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:44.548119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:44.548322 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.548440 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.548635 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":112,"threads_started":1}
I20260810 06:35:44.549338 963 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42777
I20260810 06:35:44.549427 963 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:44.549630 963 heartbeater.cc:511] Master 127.30.71.62:42777 requested a full tablet report, sending...
I20260810 06:35:44.549991 447 ts_manager.cc:194] Registered new tserver with Master: 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455)
I20260810 06:35:44.550489 447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:42039
I20260810 06:35:44.555974 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.550520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:38101 (local address 127.30.71.62:42777)
0810 06:35:44.550577 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.550579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.550728 (+ 149us) server_negotiation.cc:407] Connection header received
0810 06:35:44.550809 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.550810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.550820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.550883 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.551264 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.551736 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.552437 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.552644 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.552727 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.553617 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.553618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.553618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.553625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.554732 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.555170 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.555172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.555172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.555233 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.555444 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.555445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.555446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.555559 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.555634 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.555805 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.555884 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:35:44.555981 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.550412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42777 (local address 127.30.71.3:38101)
0810 06:35:44.550645 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.550656 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.550751 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.550899 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.550900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.551122 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:44.551172 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.551174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.551833 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.551834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.552341 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.552342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.552729 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.552730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.552820 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.553232 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:44.553236 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:44.553512 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:44.554834 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.554835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.554835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.555079 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.555309 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.555309 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.555310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.555371 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.555626 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:44.555627 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:44.555673 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.555774 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.555886 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":122,"threads_started":1}
I20260810 06:35:44.560108 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:44.586581 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.578753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57822 (local address 127.30.71.62:42777)
0810 06:35:44.578834 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.578837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.578850 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:35:44.578884 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.578885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.578904 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.578972 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.579617 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.580162 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.580913 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.581107 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.581197 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.583739 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.583741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.583741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.583750 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.585206 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.585631 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.585633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.585634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.585705 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.585932 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.585933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.585934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.586070 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.586209 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.586371 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.586466 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:35:44.588442 447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:44.590193 447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:44.597302 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.592795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46455 (local address 127.0.0.1:35578)
0810 06:35:44.592971 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.593010 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.593094 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.593741 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.593745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.593748 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:44.593833 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.593836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.594786 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.594788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.596101 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.596103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.596985 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.596986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.597071 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.597085 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.597158 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.597214 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260810 06:35:44.597365 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.593252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35578 (local address 127.30.71.3:46455)
0810 06:35:44.593511 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.593516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.593526 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:44.593568 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.593570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.593587 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.593918 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:44.593992 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.594690 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.596197 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.596888 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.596970 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.597017 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.597192 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.597246 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":108,"threads_started":1}
I20260810 06:35:44.597515 984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.592951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38532 (local address 127.30.71.2:43919)
0810 06:35:44.593501 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.593506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.593518 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:35:44.593565 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.593568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.593586 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.593952 (+ 366us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:44.594020 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.595190 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.596394 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.596991 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.597067 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.597127 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.597352 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.597404 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":171,"threads_started":1}
I20260810 06:35:44.598330 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.592795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43919 (local address 127.0.0.1:38532)
0810 06:35:44.592968 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.593010 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.593076 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.593746 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.593747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.593750 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:44.593840 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.593842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.595286 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.595287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.596302 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.596303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.597089 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.597090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.597171 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.597176 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.597241 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.597283 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":6,"thread_start_us":58,"threads_started":1}
I20260810 06:35:44.598481 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.593022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.0.0.1:35384)
0810 06:35:44.593306 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.593315 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.593489 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.593840 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.593841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.593843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:44.593880 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.593881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.595305 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.595306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.596284 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.596287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.598166 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.598167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.598247 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.598251 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.598329 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.598384 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260810 06:35:44.598378 889 tablet_service.cc:1467] Processing CreateTablet for tablet 2666f01680a144bcacaa1d94799d5aab (DEFAULT_TABLE table=test-table [id=2b035af945254fcf8183342f6578c869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:44.598814 889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2666f01680a144bcacaa1d94799d5aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.598892 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.593314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.30.71.1:44041)
0810 06:35:44.593602 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.593606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.593619 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:35:44.593659 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.593661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.593677 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.593738 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:44.593969 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.595190 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.597169 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.598091 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.598164 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.598228 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.598349 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.598783 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":93,"threads_started":1}
I20260810 06:35:44.599150 745 tablet_service.cc:1467] Processing CreateTablet for tablet 2666f01680a144bcacaa1d94799d5aab (DEFAULT_TABLE table=test-table [id=2b035af945254fcf8183342f6578c869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:44.599545 745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2666f01680a144bcacaa1d94799d5aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.599602 575 tablet_service.cc:1467] Processing CreateTablet for tablet 2666f01680a144bcacaa1d94799d5aab (DEFAULT_TABLE table=test-table [id=2b035af945254fcf8183342f6578c869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:44.599953 575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2666f01680a144bcacaa1d94799d5aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.602486 987 tablet_bootstrap.cc:492] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Bootstrap starting.
I20260810 06:35:44.603147 988 tablet_bootstrap.cc:492] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Bootstrap starting.
I20260810 06:35:44.603678 987 tablet_bootstrap.cc:654] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.603682 989 tablet_bootstrap.cc:492] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Bootstrap starting.
I20260810 06:35:44.603818 988 tablet_bootstrap.cc:654] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.604080 987 log.cc:824] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:44.604118 988 log.cc:824] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:44.604434 989 tablet_bootstrap.cc:654] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.604815 989 log.cc:824] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:44.604948 988 tablet_bootstrap.cc:492] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: No bootstrap required, opened a new log
I20260810 06:35:44.605022 988 ts_tablet_manager.cc:1397] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:35:44.605115 987 tablet_bootstrap.cc:492] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: No bootstrap required, opened a new log
I20260810 06:35:44.605178 987 ts_tablet_manager.cc:1397] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:35:44.605563 989 tablet_bootstrap.cc:492] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: No bootstrap required, opened a new log
I20260810 06:35:44.605671 989 ts_tablet_manager.cc:1397] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:35:44.607118 988 raft_consensus.cc:348] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.607178 987 raft_consensus.cc:348] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.607312 987 raft_consensus.cc:374] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.607303 988 raft_consensus.cc:374] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.607350 987 raft_consensus.cc:729] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224257b2965242a58807c242ef04a1cc, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.607362 988 raft_consensus.cc:729] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47008aebce6a49ecbc21204b7f13903d, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.607197 989 raft_consensus.cc:348] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.607414 989 raft_consensus.cc:374] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.607455 989 raft_consensus.cc:729] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.607472 988 consensus_queue.cc:260] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.607472 987 consensus_queue.cc:260] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.607576 989 consensus_queue.cc:260] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.607764 987 ts_tablet_manager.cc:1428] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:35:44.607764 988 ts_tablet_manager.cc:1428] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:35:44.607897 963 heartbeater.cc:503] Master 127.30.71.62:42777 was elected leader, sending a full tablet report...
I20260810 06:35:44.607923 989 ts_tablet_manager.cc:1428] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:35:44.608057 665 heartbeater.cc:503] Master 127.30.71.62:42777 was elected leader, sending a full tablet report...
I20260810 06:35:44.608283 815 heartbeater.cc:503] Master 127.30.71.62:42777 was elected leader, sending a full tablet report...
W20260810 06:35:44.619971 816 tablet.cc:2369] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:35:44.696081 666 tablet.cc:2369] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:35:44.709038 993 raft_consensus.cc:482] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:44.709262 993 raft_consensus.cc:504] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.709671 993 leader_election.cc:288] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455), 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041)
I20260810 06:35:44.712689 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.709980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:39273 (local address 127.30.71.3:46455)
0810 06:35:44.710028 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.710030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.710046 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:35:44.710137 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.710138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.710152 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.710218 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:44.710442 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.710956 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.711921 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.712329 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.712418 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.712428 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.712580 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.712617 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:35:44.712689 699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.709913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46455 (local address 127.30.71.2:39273)
0810 06:35:44.709984 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.709998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.710052 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.710279 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.710280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.710282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:44.710358 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.710360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.711073 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.711074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.711818 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.711820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.712416 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.712417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.712502 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.712505 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.712574 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.712617 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:35:44.713016 913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2666f01680a144bcacaa1d94799d5aab" candidate_uuid: "47008aebce6a49ecbc21204b7f13903d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc" is_pre_election: true
I20260810 06:35:44.713181 913 raft_consensus.cc:2454] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47008aebce6a49ecbc21204b7f13903d in term 0.
I20260810 06:35:44.713436 693 leader_election.cc:302] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 224257b2965242a58807c242ef04a1cc, 47008aebce6a49ecbc21204b7f13903d; no voters:
I20260810 06:35:44.713491 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.710174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:36107 (local address 127.30.71.1:44041)
0810 06:35:44.710239 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.710241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.710413 (+ 172us) server_negotiation.cc:407] Connection header received
0810 06:35:44.710514 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.710515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.710529 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.710598 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:44.710824 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.711435 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.712469 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.713006 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.713115 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.713127 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.713363 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.713409 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:35:44.713514 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.710078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.30.71.2:36107)
0810 06:35:44.710333 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.710344 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.710427 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.710620 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.710621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.710624 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:44.710725 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.710727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.711534 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.711536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.712341 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.712343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.713141 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.713143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.713228 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.713231 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.713352 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.713400 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":195,"threads_started":1}
I20260810 06:35:44.713658 993 raft_consensus.cc:2781] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:44.713728 993 raft_consensus.cc:482] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:44.713752 993 raft_consensus.cc:3037] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:44.713827 615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2666f01680a144bcacaa1d94799d5aab" candidate_uuid: "47008aebce6a49ecbc21204b7f13903d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b530c78fb948d696a93bbb2f8afcdc" is_pre_election: true
I20260810 06:35:44.714015 615 raft_consensus.cc:2454] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47008aebce6a49ecbc21204b7f13903d in term 0.
I20260810 06:35:44.714895 993 raft_consensus.cc:504] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.715070 993 leader_election.cc:288] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [CANDIDATE]: Term 1 election: Requested vote from peers 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455), 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041)
I20260810 06:35:44.715317 615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2666f01680a144bcacaa1d94799d5aab" candidate_uuid: "47008aebce6a49ecbc21204b7f13903d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b530c78fb948d696a93bbb2f8afcdc"
I20260810 06:35:44.715341 913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2666f01680a144bcacaa1d94799d5aab" candidate_uuid: "47008aebce6a49ecbc21204b7f13903d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc"
I20260810 06:35:44.715433 913 raft_consensus.cc:3037] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:44.715441 615 raft_consensus.cc:3037] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:44.716264 913 raft_consensus.cc:2454] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47008aebce6a49ecbc21204b7f13903d in term 1.
I20260810 06:35:44.716403 615 raft_consensus.cc:2454] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47008aebce6a49ecbc21204b7f13903d in term 1.
I20260810 06:35:44.716565 693 leader_election.cc:302] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 224257b2965242a58807c242ef04a1cc, 47008aebce6a49ecbc21204b7f13903d; no voters:
I20260810 06:35:44.716667 993 raft_consensus.cc:2781] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:44.716830 993 raft_consensus.cc:686] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 1 LEADER]: Becoming Leader. State: Replica: 47008aebce6a49ecbc21204b7f13903d, State: Running, Role: LEADER
I20260810 06:35:44.716940 993 consensus_queue.cc:237] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } }
I20260810 06:35:44.717844 447 catalog_manager.cc:5495] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d reported cstate change: term changed from 0 to 1, leader changed from <none> to 47008aebce6a49ecbc21204b7f13903d (127.30.71.2). New cstate: current_term: 1 leader_uuid: "47008aebce6a49ecbc21204b7f13903d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } health_report { overall_health: HEALTHY } } }
Aug 10 06:35:44 dist-test-slave-w206 krb5kdc[399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343743, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:44.777637 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.763928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.30.71.62:42777)
0810 06:35:44.764023 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.764026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.764139 (+ 113us) server_negotiation.cc:407] Connection header received
0810 06:35:44.764267 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.764268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.764282 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.764348 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:44.764856 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.765354 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.766220 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.766408 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.766493 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.774574 (+ 8081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:44.774576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:44.774576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:44.774591 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:44.776325 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.776754 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.776755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.776755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.776821 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:44.777022 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:44.777022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:44.777023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:44.777142 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:44.777221 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.777418 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.777539 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":64}
I20260810 06:35:44.777637 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.763816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42777 (local address 127.0.0.1:57828)
0810 06:35:44.764053 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.764068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.764161 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.764355 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.764356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.764716 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:44.764766 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.764768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.765440 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.765441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.766125 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.766127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.766494 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.766495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.766549 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.772820 (+ 6271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:44.772826 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:44.774446 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:44.776438 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.776450 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.776453 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.776661 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.776889 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:44.776890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:44.776890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:44.776957 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:44.777211 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:44.777221 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:44.777272 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.777354 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.777489 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":138,"threads_started":1}
I20260810 06:35:44.780216 447 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:57828:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:35:44.780496 447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:44.782176 575 tablet_service.cc:1467] Processing CreateTablet for tablet 69c408df77364f50bfe3a5b21d394997 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c2159e8fc0624b56943d048a3039c383]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:35:44.782377 575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c408df77364f50bfe3a5b21d394997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.782325 889 tablet_service.cc:1467] Processing CreateTablet for tablet 69c408df77364f50bfe3a5b21d394997 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c2159e8fc0624b56943d048a3039c383]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:35:44.782322 745 tablet_service.cc:1467] Processing CreateTablet for tablet 69c408df77364f50bfe3a5b21d394997 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c2159e8fc0624b56943d048a3039c383]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:35:44.782482 889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c408df77364f50bfe3a5b21d394997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.782519 745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c408df77364f50bfe3a5b21d394997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.784075 989 tablet_bootstrap.cc:492] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: Bootstrap starting.
I20260810 06:35:44.784255 988 tablet_bootstrap.cc:492] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: Bootstrap starting.
I20260810 06:35:44.784593 989 tablet_bootstrap.cc:654] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.784647 988 tablet_bootstrap.cc:654] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.784935 987 tablet_bootstrap.cc:492] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: Bootstrap starting.
I20260810 06:35:44.785292 989 tablet_bootstrap.cc:492] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: No bootstrap required, opened a new log
I20260810 06:35:44.785368 989 ts_tablet_manager.cc:1397] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:44.785542 988 tablet_bootstrap.cc:492] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: No bootstrap required, opened a new log
I20260810 06:35:44.785581 988 ts_tablet_manager.cc:1397] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:44.785540 989 raft_consensus.cc:348] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.785619 989 raft_consensus.cc:374] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.785645 989 raft_consensus.cc:729] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.785642 987 tablet_bootstrap.cc:654] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.785699 989 consensus_queue.cc:260] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.785722 988 raft_consensus.cc:348] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.785821 988 raft_consensus.cc:374] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.785820 989 ts_tablet_manager.cc:1428] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.785844 988 raft_consensus.cc:729] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47008aebce6a49ecbc21204b7f13903d, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.785893 988 consensus_queue.cc:260] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.785983 988 ts_tablet_manager.cc:1428] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:44.785887 995 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.786031 995 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:44.786048 993 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.786119 993 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:44.786639 987 tablet_bootstrap.cc:492] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: No bootstrap required, opened a new log
I20260810 06:35:44.786710 987 ts_tablet_manager.cc:1397] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:35:44.786906 987 raft_consensus.cc:348] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.787025 987 raft_consensus.cc:374] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.787063 987 raft_consensus.cc:729] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224257b2965242a58807c242ef04a1cc, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.787128 987 consensus_queue.cc:260] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.787261 987 ts_tablet_manager.cc:1428] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:44.787457 994 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.787585 994 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 06:35:44.797452 965 tablet.cc:2369] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:35:44.906275 995 raft_consensus.cc:482] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:44.906399 995 raft_consensus.cc:504] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.906869 995 leader_election.cc:288] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919), 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455)
I20260810 06:35:44.911042 984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.907192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:55737 (local address 127.30.71.2:43919)
0810 06:35:44.907283 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.907285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.907380 (+ 95us) server_negotiation.cc:407] Connection header received
0810 06:35:44.907484 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.907485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.907498 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.907572 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:44.907780 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.908365 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.910221 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.910647 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.910729 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.910740 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.910906 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.910946 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:35:44.911059 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.907120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43919 (local address 127.30.71.1:55737)
0810 06:35:44.907303 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.907316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.907397 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.907591 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.907592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.907595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:44.907692 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.907694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.908495 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.908496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.910101 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.910105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.910769 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.910770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.910826 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.910835 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.910885 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.910919 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
I20260810 06:35:44.911638 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.907141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46455 (local address 127.30.71.1:58605)
0810 06:35:44.907344 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.907354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.907416 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.907639 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.907641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.907643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:44.907773 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.907776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.908613 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.908614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.910483 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.910486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.911394 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.911396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.911475 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.911479 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.911529 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.911558 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":12,"thread_start_us":139,"threads_started":1}
I20260810 06:35:44.911691 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c408df77364f50bfe3a5b21d394997" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d" is_pre_election: true
I20260810 06:35:44.911806 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.907272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:58605 (local address 127.30.71.3:46455)
0810 06:35:44.907330 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.907333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.907452 (+ 119us) server_negotiation.cc:407] Connection header received
0810 06:35:44.907484 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.907485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.907496 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.907583 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:44.907848 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.908545 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.910581 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.911176 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.911247 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.911257 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.911670 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.911739 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:35:44.911844 766 raft_consensus.cc:2454] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 0.
I20260810 06:35:44.912056 913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c408df77364f50bfe3a5b21d394997" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc" is_pre_election: true
I20260810 06:35:44.912082 541 leader_election.cc:302] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47008aebce6a49ecbc21204b7f13903d, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:44.912160 913 raft_consensus.cc:2454] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 0.
I20260810 06:35:44.912245 995 raft_consensus.cc:2781] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:44.912308 995 raft_consensus.cc:482] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:44.912344 995 raft_consensus.cc:3037] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:44.913118 995 raft_consensus.cc:504] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.913300 995 leader_election.cc:288] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 election: Requested vote from peers 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919), 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455)
I20260810 06:35:44.913467 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c408df77364f50bfe3a5b21d394997" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d"
I20260810 06:35:44.913494 913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c408df77364f50bfe3a5b21d394997" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc"
I20260810 06:35:44.913568 913 raft_consensus.cc:3037] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:44.913571 766 raft_consensus.cc:3037] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:44.914238 913 raft_consensus.cc:2454] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 1.
I20260810 06:35:44.914250 766 raft_consensus.cc:2454] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 1.
I20260810 06:35:44.914412 543 leader_election.cc:302] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 224257b2965242a58807c242ef04a1cc, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:44.914546 995 raft_consensus.cc:2781] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:44.914672 995 raft_consensus.cc:686] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 LEADER]: Becoming Leader. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Running, Role: LEADER
I20260810 06:35:44.914789 995 consensus_queue.cc:237] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.915479 1011 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: TxnStatusTablet state changed. Reason: New leader 77b530c78fb948d696a93bbb2f8afcdc. Latest consensus state: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.915572 1011 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:35:44.915759 1013 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:35:44.915903 1013 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:35:44.915971 448 catalog_manager.cc:5495] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1). New cstate: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } health_report { overall_health: UNKNOWN } } }
I20260810 06:35:44.933907 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.930927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35388 (local address 127.30.71.1:44041)
0810 06:35:44.931006 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.931009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.931020 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:35:44.931126 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.931127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.931166 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.931257 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:44.931423 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.932000 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.932935 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.933166 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.933261 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.933494 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:44.933611 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:44.933722 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.933786 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.933814 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 06:35:44.933917 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.930831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.0.0.1:35388)
0810 06:35:44.930920 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.930934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.931021 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.931262 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.931263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.931264 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:44.931323 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.931326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.932150 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.932151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.932809 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.932812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.933260 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.933261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.933342 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.933365 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:44.933698 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:44.933710 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.933781 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.933838 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:35:44.936264 766 raft_consensus.cc:1261] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Refusing update from remote peer 77b530c78fb948d696a93bbb2f8afcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:44.936246 913 raft_consensus.cc:1261] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Refusing update from remote peer 77b530c78fb948d696a93bbb2f8afcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:44.936600 1011 consensus_queue.cc:1035] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:44.936753 995 consensus_queue.cc:1035] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:44.938266 993 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: TxnStatusTablet state changed. Reason: New leader 77b530c78fb948d696a93bbb2f8afcdc. Latest consensus state: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.938386 993 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:44.939596 995 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.939633 994 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: TxnStatusTablet state changed. Reason: New leader 77b530c78fb948d696a93bbb2f8afcdc. Latest consensus state: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.939719 995 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:35:44.939752 994 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:44.939802 994 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.939845 994 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 224257b2965242a58807c242ef04a1cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:44.939986 1016 mvcc.cc:205] Tried to move back new op lower bound from 7316863979255758848 to 7316863979171954688. Current Snapshot: MvccSnapshot[applied={T|T < 7316863979255758848}]
I20260810 06:35:44.940698 1018 mvcc.cc:205] Tried to move back new op lower bound from 7316863979255758848 to 7316863979171954688. Current Snapshot: MvccSnapshot[applied={T|T < 7316863979255758848 or (T in {7316863979255758848})}]
I20260810 06:35:44.943667 1011 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.943783 1011 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 77b530c78fb948d696a93bbb2f8afcdc: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:35:44.944422 993 tablet_replica.cc:439] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } } }
I20260810 06:35:44.944520 993 tablet_replica.cc:442] T 69c408df77364f50bfe3a5b21d394997 P 47008aebce6a49ecbc21204b7f13903d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:35:44.944746 1017 mvcc.cc:205] Tried to move back new op lower bound from 7316863979255758848 to 7316863979171954688. Current Snapshot: MvccSnapshot[applied={T|T < 7316863979255758848}]
I20260810 06:35:44.946075 984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.942644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38536 (local address 127.30.71.2:43919)
0810 06:35:44.942694 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.942697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.942719 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:35:44.942811 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.942811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.942829 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.942892 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:44.943092 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.943961 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.945143 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.945431 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.945519 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.945715 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:44.945851 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:44.945946 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.945975 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.945998 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:35:44.951797 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.948893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:33889 (local address 127.30.71.1:44041)
0810 06:35:44.948936 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.948938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.949096 (+ 158us) server_negotiation.cc:407] Connection header received
0810 06:35:44.949182 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.949183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.949196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.949258 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:44.949419 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.950014 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.950880 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.951186 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.951279 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.951503 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:44.951546 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:44.951613 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.951689 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.951718 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:35:44.951820 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.948819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.30.71.2:33889)
0810 06:35:44.949020 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.949031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.949113 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.949260 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.949261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.949263 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:44.949330 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.949331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.950109 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.950110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.950782 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.950784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.951282 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.951283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.951371 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.951386 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:44.951613 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:44.951616 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.951683 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.951728 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260810 06:35:44.954463 615 raft_consensus.cc:1261] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Refusing update from remote peer 47008aebce6a49ecbc21204b7f13903d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:44.954694 913 raft_consensus.cc:1261] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Refusing update from remote peer 47008aebce6a49ecbc21204b7f13903d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:44.954845 997 consensus_queue.cc:1035] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [LEADER]: Connected to new peer: Peer: permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:44.955001 993 consensus_queue.cc:1035] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [LEADER]: Connected to new peer: Peer: permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:44.965304 984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.962140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:59381 (local address 127.30.71.2:43919)
0810 06:35:44.962204 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:44.962206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:44.962341 (+ 135us) server_negotiation.cc:407] Connection header received
0810 06:35:44.962372 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:44.962373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:44.962385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:44.962457 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:44.962564 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.963408 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.964465 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.964811 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.964889 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.965054 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:44.965090 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:44.965153 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:44.965200 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 06:35:44.965221 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:35:44.965310 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:44.962055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43919 (local address 127.30.71.1:59381)
0810 06:35:44.962259 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:44.962271 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:44.962336 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:44.962457 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:44.962458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:44.962459 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:44.962523 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.962524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.963482 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.963483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.964374 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:44.964376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:44.964894 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:44.964895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:44.964949 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:44.964964 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:44.965153 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:44.965154 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:35:44.965197 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 06:35:44.965232 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":126,"threads_started":1}
I20260810 06:35:44.974278 448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:44.974403 448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:44.975481 448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Sending DeleteTablet for 3 replicas of tablet 2666f01680a144bcacaa1d94799d5aab
I20260810 06:35:44.975927 889 tablet_service.cc:1514] Processing DeleteTablet for tablet 2666f01680a144bcacaa1d94799d5aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:44 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35578
I20260810 06:35:44.976256 1037 tablet_replica.cc:330] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: stopping tablet replica
I20260810 06:35:44.976461 1037 raft_consensus.cc:2229] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:44.976601 1037 raft_consensus.cc:2258] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:44.976624 575 tablet_service.cc:1514] Processing DeleteTablet for tablet 2666f01680a144bcacaa1d94799d5aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:44 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35384
I20260810 06:35:44.976949 1038 tablet_replica.cc:330] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: stopping tablet replica
I20260810 06:35:44.976964 448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:44.977130 448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:44.977355 1038 raft_consensus.cc:2229] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:44.977492 1038 raft_consensus.cc:2258] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:44.977836 745 tablet_service.cc:1514] Processing DeleteTablet for tablet 2666f01680a144bcacaa1d94799d5aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:44 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:38532
I20260810 06:35:44.978242 1038 ts_tablet_manager.cc:1905] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:44.979022 889 tablet_service.cc:1467] Processing CreateTablet for tablet 5152309091ce4f27a6cd9d3fa567da36 (DEFAULT_TABLE table=test-table [id=0ca750030a7e4e7d99885469757cb9a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:44.979171 889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5152309091ce4f27a6cd9d3fa567da36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.980613 1038 ts_tablet_manager.cc:1918] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:44.980681 987 tablet_bootstrap.cc:492] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: Bootstrap starting.
I20260810 06:35:44.980813 1038 log.cc:1196] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/2666f01680a144bcacaa1d94799d5aab
I20260810 06:35:44.981117 575 tablet_service.cc:1467] Processing CreateTablet for tablet 5152309091ce4f27a6cd9d3fa567da36 (DEFAULT_TABLE table=test-table [id=0ca750030a7e4e7d99885469757cb9a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:44.981176 987 tablet_bootstrap.cc:654] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.981246 575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5152309091ce4f27a6cd9d3fa567da36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.981453 1038 ts_tablet_manager.cc:1939] T 2666f01680a144bcacaa1d94799d5aab P 77b530c78fb948d696a93bbb2f8afcdc: Deleting consensus metadata
I20260810 06:35:44.981997 432 catalog_manager.cc:4841] TS 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041): tablet 2666f01680a144bcacaa1d94799d5aab (table test-table [id=2b035af945254fcf8183342f6578c869]) successfully deleted
I20260810 06:35:44.982554 989 tablet_bootstrap.cc:492] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: Bootstrap starting.
I20260810 06:35:44.983055 989 tablet_bootstrap.cc:654] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.983223 1039 tablet_replica.cc:330] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: stopping tablet replica
I20260810 06:35:44.983402 1039 raft_consensus.cc:2229] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:44.983501 745 tablet_service.cc:1467] Processing CreateTablet for tablet 5152309091ce4f27a6cd9d3fa567da36 (DEFAULT_TABLE table=test-table [id=0ca750030a7e4e7d99885469757cb9a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:44.983630 745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5152309091ce4f27a6cd9d3fa567da36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:44.984234 1037 ts_tablet_manager.cc:1905] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:44.985209 988 tablet_bootstrap.cc:492] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: Bootstrap starting.
I20260810 06:35:44.985658 988 tablet_bootstrap.cc:654] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:44.983621 1039 raft_consensus.cc:2258] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:44.986763 987 tablet_bootstrap.cc:492] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: No bootstrap required, opened a new log
I20260810 06:35:44.986822 987 ts_tablet_manager.cc:1397] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260810 06:35:44.986841 1039 ts_tablet_manager.cc:1905] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:44.987023 987 raft_consensus.cc:348] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.987128 987 raft_consensus.cc:374] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.987156 987 raft_consensus.cc:729] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224257b2965242a58807c242ef04a1cc, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.987218 987 consensus_queue.cc:260] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.987350 987 ts_tablet_manager.cc:1428] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:44.986334 1037 ts_tablet_manager.cc:1918] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:44.988540 1037 log.cc:1196] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/2666f01680a144bcacaa1d94799d5aab
I20260810 06:35:44.988869 1037 ts_tablet_manager.cc:1939] T 2666f01680a144bcacaa1d94799d5aab P 224257b2965242a58807c242ef04a1cc: Deleting consensus metadata
I20260810 06:35:44.989321 432 catalog_manager.cc:4841] TS 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455): tablet 2666f01680a144bcacaa1d94799d5aab (table test-table [id=2b035af945254fcf8183342f6578c869]) successfully deleted
I20260810 06:35:44.990085 1039 ts_tablet_manager.cc:1918] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:44.990170 1039 log.cc:1196] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/2666f01680a144bcacaa1d94799d5aab
I20260810 06:35:44.990464 1039 ts_tablet_manager.cc:1939] T 2666f01680a144bcacaa1d94799d5aab P 47008aebce6a49ecbc21204b7f13903d: Deleting consensus metadata
I20260810 06:35:44.990938 429 catalog_manager.cc:4841] TS 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919): tablet 2666f01680a144bcacaa1d94799d5aab (table test-table [id=2b035af945254fcf8183342f6578c869]) successfully deleted
I20260810 06:35:44.991218 988 tablet_bootstrap.cc:492] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: No bootstrap required, opened a new log
I20260810 06:35:44.991278 988 ts_tablet_manager.cc:1397] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260810 06:35:44.991441 988 raft_consensus.cc:348] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.991523 988 raft_consensus.cc:374] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.991546 988 raft_consensus.cc:729] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47008aebce6a49ecbc21204b7f13903d, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.991602 988 consensus_queue.cc:260] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.991698 988 ts_tablet_manager.cc:1428] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:44.995545 989 tablet_bootstrap.cc:492] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: No bootstrap required, opened a new log
I20260810 06:35:44.995627 989 ts_tablet_manager.cc:1397] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20260810 06:35:44.995782 989 raft_consensus.cc:348] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.995872 989 raft_consensus.cc:374] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:44.995895 989 raft_consensus.cc:729] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Initialized, Role: FOLLOWER
I20260810 06:35:44.995954 989 consensus_queue.cc:260] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.996052 989 ts_tablet_manager.cc:1428] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 06:35:44.998400 1032 raft_consensus.cc:482] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:44.998487 1032 raft_consensus.cc:504] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:44.998889 913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5152309091ce4f27a6cd9d3fa567da36" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc" is_pre_election: true
I20260810 06:35:44.998910 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5152309091ce4f27a6cd9d3fa567da36" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d" is_pre_election: true
I20260810 06:35:44.999012 913 raft_consensus.cc:2454] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 0.
I20260810 06:35:44.999037 766 raft_consensus.cc:2454] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 0.
I20260810 06:35:44.999231 543 leader_election.cc:302] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 224257b2965242a58807c242ef04a1cc, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:44.999336 1011 raft_consensus.cc:2781] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:44.999373 1011 raft_consensus.cc:482] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:44.999402 1011 raft_consensus.cc:3037] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:45.000357 1011 raft_consensus.cc:504] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:45.000511 1011 leader_election.cc:288] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 election: Requested vote from peers 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919), 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455)
I20260810 06:35:45.000900 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5152309091ce4f27a6cd9d3fa567da36" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d"
I20260810 06:35:45.000998 766 raft_consensus.cc:3037] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:45.001883 766 raft_consensus.cc:2454] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 1.
I20260810 06:35:45.002079 541 leader_election.cc:302] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47008aebce6a49ecbc21204b7f13903d, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:45.002218 913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5152309091ce4f27a6cd9d3fa567da36" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc"
I20260810 06:35:45.002311 913 raft_consensus.cc:3037] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:45.003113 1011 raft_consensus.cc:2781] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:45.003180 1011 raft_consensus.cc:686] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 LEADER]: Becoming Leader. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Running, Role: LEADER
I20260810 06:35:45.003232 913 raft_consensus.cc:2454] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 1.
I20260810 06:35:45.003228 1011 consensus_queue.cc:237] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:45.003846 1032 leader_election.cc:288] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919), 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455)
I20260810 06:35:45.003800 447 catalog_manager.cc:5495] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1). New cstate: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } health_report { overall_health: UNKNOWN } } }
I20260810 06:35:45.018386 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:45.014869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.30.71.1:44041)
0810 06:35:45.014940 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:45.014942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:45.014951 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:35:45.015013 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:45.015013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:45.015025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:45.015088 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:45.015321 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:45.016144 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:45.017379 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:45.017718 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:45.017786 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:45.017979 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:45.018019 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:45.018066 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:45.018273 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 06:35:45.018294 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:35:45.024559 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:45.021303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:53999 (local address 127.30.71.1:44041)
0810 06:35:45.021340 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:45.021342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:45.021352 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:45.021404 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:45.021405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:45.021415 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:45.021478 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:45.021613 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:45.022342 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:45.023613 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:45.023965 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:45.024049 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:45.024225 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:45.024263 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:45.024329 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:45.024406 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 06:35:45.024429 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 06:35:45.024569 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:45.021211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.30.71.1:53999)
0810 06:35:45.021274 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:45.021288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:45.021342 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:45.021491 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:45.021492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:45.021493 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:45.021539 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:45.021540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:45.022438 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:45.022439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:45.023506 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:45.023510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:45.024050 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:45.024051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:45.024116 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:45.024124 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:45.024347 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:45.024350 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:45.024403 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 06:35:45.024452 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:35:45.027486 913 raft_consensus.cc:1261] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Refusing update from remote peer 77b530c78fb948d696a93bbb2f8afcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:45.027743 766 raft_consensus.cc:1261] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Refusing update from remote peer 77b530c78fb948d696a93bbb2f8afcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:45.027756 1032 consensus_queue.cc:1035] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:45.027954 1011 consensus_queue.cc:1035] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:35:45.035255 1016 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:35:46.568663 448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:46.568787 448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:46.569804 448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Sending DeleteTablet for 3 replicas of tablet 5152309091ce4f27a6cd9d3fa567da36
I20260810 06:35:46.570153 575 tablet_service.cc:1514] Processing DeleteTablet for tablet 5152309091ce4f27a6cd9d3fa567da36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:46 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35384
I20260810 06:35:46.570180 745 tablet_service.cc:1514] Processing DeleteTablet for tablet 5152309091ce4f27a6cd9d3fa567da36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:46 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:38532
I20260810 06:35:46.570176 889 tablet_service.cc:1514] Processing DeleteTablet for tablet 5152309091ce4f27a6cd9d3fa567da36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:46 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35578
I20260810 06:35:46.570395 1069 tablet_replica.cc:330] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: stopping tablet replica
I20260810 06:35:46.570403 1068 tablet_replica.cc:330] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: stopping tablet replica
I20260810 06:35:46.570456 1068 raft_consensus.cc:2229] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:46.570456 1069 raft_consensus.cc:2229] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:46.570503 1068 raft_consensus.cc:2258] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:46.570554 1069 raft_consensus.cc:2258] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:46.570770 1068 ts_tablet_manager.cc:1905] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:46.571075 1070 tablet_replica.cc:330] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: stopping tablet replica
I20260810 06:35:46.571137 1070 raft_consensus.cc:2229] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:46.571177 1070 raft_consensus.cc:2258] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:46.572589 1068 ts_tablet_manager.cc:1918] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:35:46.572661 1068 log.cc:1196] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/5152309091ce4f27a6cd9d3fa567da36
I20260810 06:35:46.572909 1068 ts_tablet_manager.cc:1939] T 5152309091ce4f27a6cd9d3fa567da36 P 47008aebce6a49ecbc21204b7f13903d: Deleting consensus metadata
I20260810 06:35:46.573195 1069 ts_tablet_manager.cc:1905] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:46.573329 429 catalog_manager.cc:4841] TS 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919): tablet 5152309091ce4f27a6cd9d3fa567da36 (table test-table [id=0ca750030a7e4e7d99885469757cb9a7]) successfully deleted
I20260810 06:35:46.574295 1070 ts_tablet_manager.cc:1905] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:46.575441 1069 ts_tablet_manager.cc:1918] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:35:46.575512 1069 log.cc:1196] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/5152309091ce4f27a6cd9d3fa567da36
I20260810 06:35:46.575788 1069 ts_tablet_manager.cc:1939] T 5152309091ce4f27a6cd9d3fa567da36 P 77b530c78fb948d696a93bbb2f8afcdc: Deleting consensus metadata
I20260810 06:35:46.576150 1070 ts_tablet_manager.cc:1918] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:35:46.576138 432 catalog_manager.cc:4841] TS 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041): tablet 5152309091ce4f27a6cd9d3fa567da36 (table test-table [id=0ca750030a7e4e7d99885469757cb9a7]) successfully deleted
I20260810 06:35:46.576205 1070 log.cc:1196] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/5152309091ce4f27a6cd9d3fa567da36
I20260810 06:35:46.576453 1070 ts_tablet_manager.cc:1939] T 5152309091ce4f27a6cd9d3fa567da36 P 224257b2965242a58807c242ef04a1cc: Deleting consensus metadata
I20260810 06:35:46.576869 432 catalog_manager.cc:4841] TS 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455): tablet 5152309091ce4f27a6cd9d3fa567da36 (table test-table [id=0ca750030a7e4e7d99885469757cb9a7]) successfully deleted
Aug 10 06:35:46 dist-test-slave-w206 krb5kdc[399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343746, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:35:46 dist-test-slave-w206 krb5kdc[399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343746, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:46.599298 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.591667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.30.71.62:42777)
0810 06:35:46.591840 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:46.591842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:46.591875 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:35:46.591899 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:46.591900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:46.591910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:46.591963 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:46.592551 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.593066 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.593861 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.594092 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.594207 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.596529 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:46.596530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:46.596530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:46.596537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:46.598165 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:46.598521 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:46.598522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:46.598522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:46.598590 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:46.598772 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:46.598772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:46.598773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:46.598878 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:46.598955 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:46.599109 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 06:35:46.599226 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260810 06:35:46.600798 448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:46.600984 448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:46.602563 889 tablet_service.cc:1467] Processing CreateTablet for tablet 889c79ecb4244d8c9a9fd6e9982f70c8 (DEFAULT_TABLE table=test-table [id=d23b90c94a234ca3a5a219f6aa0a792b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:46.602605 745 tablet_service.cc:1467] Processing CreateTablet for tablet 889c79ecb4244d8c9a9fd6e9982f70c8 (DEFAULT_TABLE table=test-table [id=d23b90c94a234ca3a5a219f6aa0a792b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:46.602624 575 tablet_service.cc:1467] Processing CreateTablet for tablet 889c79ecb4244d8c9a9fd6e9982f70c8 (DEFAULT_TABLE table=test-table [id=d23b90c94a234ca3a5a219f6aa0a792b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:46.602731 745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 889c79ecb4244d8c9a9fd6e9982f70c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:46.602731 889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 889c79ecb4244d8c9a9fd6e9982f70c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:46.602810 575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 889c79ecb4244d8c9a9fd6e9982f70c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:46.604014 1080 tablet_bootstrap.cc:492] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: Bootstrap starting.
I20260810 06:35:46.604190 1081 tablet_bootstrap.cc:492] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: Bootstrap starting.
I20260810 06:35:46.604327 1082 tablet_bootstrap.cc:492] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: Bootstrap starting.
I20260810 06:35:46.604660 1081 tablet_bootstrap.cc:654] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:46.604660 1080 tablet_bootstrap.cc:654] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:46.604741 1082 tablet_bootstrap.cc:654] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:46.605340 1081 tablet_bootstrap.cc:492] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: No bootstrap required, opened a new log
I20260810 06:35:46.605403 1081 ts_tablet_manager.cc:1397] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:46.605468 1080 tablet_bootstrap.cc:492] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: No bootstrap required, opened a new log
I20260810 06:35:46.605523 1082 tablet_bootstrap.cc:492] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: No bootstrap required, opened a new log
I20260810 06:35:46.605530 1080 ts_tablet_manager.cc:1397] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:35:46.605526 1081 raft_consensus.cc:348] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.605577 1082 ts_tablet_manager.cc:1397] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:46.605608 1081 raft_consensus.cc:374] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:46.605638 1081 raft_consensus.cc:729] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47008aebce6a49ecbc21204b7f13903d, State: Initialized, Role: FOLLOWER
I20260810 06:35:46.605660 1080 raft_consensus.cc:348] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.605693 1082 raft_consensus.cc:348] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.605748 1080 raft_consensus.cc:374] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:46.605754 1082 raft_consensus.cc:374] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:46.605726 1081 consensus_queue.cc:260] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.605780 1082 raft_consensus.cc:729] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224257b2965242a58807c242ef04a1cc, State: Initialized, Role: FOLLOWER
I20260810 06:35:46.605780 1080 raft_consensus.cc:729] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Initialized, Role: FOLLOWER
I20260810 06:35:46.605824 1082 consensus_queue.cc:260] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.605827 1080 consensus_queue.cc:260] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.605932 1081 ts_tablet_manager.cc:1428] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:46.605993 1082 ts_tablet_manager.cc:1428] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:46.605988 1080 ts_tablet_manager.cc:1428] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:46.758417 1056 raft_consensus.cc:482] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:46.758520 1056 raft_consensus.cc:504] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.758733 1056 leader_election.cc:288] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919), 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455)
I20260810 06:35:46.759047 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889c79ecb4244d8c9a9fd6e9982f70c8" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d" is_pre_election: true
I20260810 06:35:46.759145 766 raft_consensus.cc:2454] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 0.
I20260810 06:35:46.759068 912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889c79ecb4244d8c9a9fd6e9982f70c8" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc" is_pre_election: true
I20260810 06:35:46.759186 912 raft_consensus.cc:2454] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 0.
I20260810 06:35:46.759318 541 leader_election.cc:302] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47008aebce6a49ecbc21204b7f13903d, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:46.759428 1056 raft_consensus.cc:2781] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:46.759481 1056 raft_consensus.cc:482] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:46.759495 1056 raft_consensus.cc:3037] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:46.760306 1056 raft_consensus.cc:504] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.760447 1056 leader_election.cc:288] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 election: Requested vote from peers 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919), 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455)
I20260810 06:35:46.760632 912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889c79ecb4244d8c9a9fd6e9982f70c8" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "224257b2965242a58807c242ef04a1cc"
I20260810 06:35:46.760623 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "889c79ecb4244d8c9a9fd6e9982f70c8" candidate_uuid: "77b530c78fb948d696a93bbb2f8afcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d"
I20260810 06:35:46.760695 912 raft_consensus.cc:3037] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:46.760695 766 raft_consensus.cc:3037] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:46.761466 912 raft_consensus.cc:2454] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 1.
I20260810 06:35:46.761466 766 raft_consensus.cc:2454] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b530c78fb948d696a93bbb2f8afcdc in term 1.
I20260810 06:35:46.761698 541 leader_election.cc:302] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47008aebce6a49ecbc21204b7f13903d, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:46.761842 1056 raft_consensus.cc:2781] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:46.761940 1056 raft_consensus.cc:686] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 LEADER]: Becoming Leader. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Running, Role: LEADER
I20260810 06:35:46.762053 1056 consensus_queue.cc:237] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.762876 445 catalog_manager.cc:5495] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1). New cstate: current_term: 1 leader_uuid: "77b530c78fb948d696a93bbb2f8afcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } health_report { overall_health: UNKNOWN } } }
I20260810 06:35:46.792588 1090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.789967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.30.71.1:44041)
0810 06:35:46.790165 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:46.790167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:46.790173 (+ 6us) server_negotiation.cc:407] Connection header received
0810 06:35:46.790191 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:46.790191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:46.790206 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:46.790288 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:46.790446 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.791048 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.791833 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.792061 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.792140 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.792304 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:46.792340 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:46.792408 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:46.792471 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:35:46.792494 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":127,"threads_started":1}
I20260810 06:35:46.795054 766 raft_consensus.cc:1261] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Refusing update from remote peer 77b530c78fb948d696a93bbb2f8afcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:46.795077 912 raft_consensus.cc:1261] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Refusing update from remote peer 77b530c78fb948d696a93bbb2f8afcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:46.795274 1011 consensus_queue.cc:1035] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:46.795387 1056 consensus_queue.cc:1035] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:46.805626 448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:46.805737 448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:46.806399 448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Sending DeleteTablet for 3 replicas of tablet 889c79ecb4244d8c9a9fd6e9982f70c8
I20260810 06:35:46.806672 575 tablet_service.cc:1514] Processing DeleteTablet for tablet 889c79ecb4244d8c9a9fd6e9982f70c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:46 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35384
I20260810 06:35:46.806680 745 tablet_service.cc:1514] Processing DeleteTablet for tablet 889c79ecb4244d8c9a9fd6e9982f70c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:46 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:38532
I20260810 06:35:46.806743 889 tablet_service.cc:1514] Processing DeleteTablet for tablet 889c79ecb4244d8c9a9fd6e9982f70c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:46 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35578
I20260810 06:35:46.806807 1068 tablet_replica.cc:330] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: stopping tablet replica
I20260810 06:35:46.806842 1070 tablet_replica.cc:330] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: stopping tablet replica
I20260810 06:35:46.806849 1069 tablet_replica.cc:330] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: stopping tablet replica
I20260810 06:35:46.806882 1068 raft_consensus.cc:2229] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:46.806898 1070 raft_consensus.cc:2229] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:46.806922 1068 raft_consensus.cc:2258] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:46.806917 1069 raft_consensus.cc:2229] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:46.806942 1070 raft_consensus.cc:2258] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:46.806999 1069 raft_consensus.cc:2258] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:46.807160 448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:46.807341 448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:46.807344 1070 ts_tablet_manager.cc:1905] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:46.807344 1069 ts_tablet_manager.cc:1905] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:46.809170 1068 ts_tablet_manager.cc:1905] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:46.809254 1070 ts_tablet_manager.cc:1918] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:46.809309 1070 log.cc:1196] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/889c79ecb4244d8c9a9fd6e9982f70c8
I20260810 06:35:46.809314 1069 ts_tablet_manager.cc:1918] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:46.809367 1069 log.cc:1196] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/889c79ecb4244d8c9a9fd6e9982f70c8
I20260810 06:35:46.809562 1070 ts_tablet_manager.cc:1939] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 224257b2965242a58807c242ef04a1cc: Deleting consensus metadata
I20260810 06:35:46.809588 1069 ts_tablet_manager.cc:1939] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 77b530c78fb948d696a93bbb2f8afcdc: Deleting consensus metadata
I20260810 06:35:46.809942 432 catalog_manager.cc:4841] TS 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455): tablet 889c79ecb4244d8c9a9fd6e9982f70c8 (table test-table [id=d23b90c94a234ca3a5a219f6aa0a792b]) successfully deleted
I20260810 06:35:46.810040 432 catalog_manager.cc:4841] TS 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041): tablet 889c79ecb4244d8c9a9fd6e9982f70c8 (table test-table [id=d23b90c94a234ca3a5a219f6aa0a792b]) successfully deleted
I20260810 06:35:46.810214 745 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb7a3b64e334c6f97b376c7ff063196 (DEFAULT_TABLE table=test-table [id=74b127316f0b426fb36e3e021d93d741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:46.810238 889 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb7a3b64e334c6f97b376c7ff063196 (DEFAULT_TABLE table=test-table [id=74b127316f0b426fb36e3e021d93d741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:46.810289 575 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb7a3b64e334c6f97b376c7ff063196 (DEFAULT_TABLE table=test-table [id=74b127316f0b426fb36e3e021d93d741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:46.810324 745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb7a3b64e334c6f97b376c7ff063196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:46.810341 889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb7a3b64e334c6f97b376c7ff063196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:46.810381 575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb7a3b64e334c6f97b376c7ff063196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:46.810842 1068 ts_tablet_manager.cc:1918] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:35:46.810902 1068 log.cc:1196] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/889c79ecb4244d8c9a9fd6e9982f70c8
I20260810 06:35:46.811152 1068 ts_tablet_manager.cc:1939] T 889c79ecb4244d8c9a9fd6e9982f70c8 P 47008aebce6a49ecbc21204b7f13903d: Deleting consensus metadata
I20260810 06:35:46.811515 1081 tablet_bootstrap.cc:492] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: Bootstrap starting.
I20260810 06:35:46.811621 429 catalog_manager.cc:4841] TS 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919): tablet 889c79ecb4244d8c9a9fd6e9982f70c8 (table test-table [id=d23b90c94a234ca3a5a219f6aa0a792b]) successfully deleted
I20260810 06:35:46.811832 1082 tablet_bootstrap.cc:492] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: Bootstrap starting.
I20260810 06:35:46.811844 1081 tablet_bootstrap.cc:654] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:46.812124 1080 tablet_bootstrap.cc:492] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc: Bootstrap starting.
I20260810 06:35:46.812299 1082 tablet_bootstrap.cc:654] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:46.812454 1081 tablet_bootstrap.cc:492] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: No bootstrap required, opened a new log
I20260810 06:35:46.812464 1080 tablet_bootstrap.cc:654] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:46.812523 1081 ts_tablet_manager.cc:1397] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:46.812642 1081 raft_consensus.cc:348] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.812757 1081 raft_consensus.cc:374] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:46.812793 1081 raft_consensus.cc:729] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47008aebce6a49ecbc21204b7f13903d, State: Initialized, Role: FOLLOWER
I20260810 06:35:46.812851 1081 consensus_queue.cc:260] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.812978 1081 ts_tablet_manager.cc:1428] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:46.813059 1082 tablet_bootstrap.cc:492] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: No bootstrap required, opened a new log
I20260810 06:35:46.813102 1082 ts_tablet_manager.cc:1397] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:46.813269 1082 raft_consensus.cc:348] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.813341 1082 raft_consensus.cc:374] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:46.813360 1082 raft_consensus.cc:729] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224257b2965242a58807c242ef04a1cc, State: Initialized, Role: FOLLOWER
I20260810 06:35:46.813398 1082 consensus_queue.cc:260] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.813509 1082 ts_tablet_manager.cc:1428] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:46.813881 1080 tablet_bootstrap.cc:492] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc: No bootstrap required, opened a new log
I20260810 06:35:46.813941 1080 ts_tablet_manager.cc:1397] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:35:46.814079 1080 raft_consensus.cc:348] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.814157 1080 raft_consensus.cc:374] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:46.814178 1080 raft_consensus.cc:729] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b530c78fb948d696a93bbb2f8afcdc, State: Initialized, Role: FOLLOWER
I20260810 06:35:46.814260 1080 consensus_queue.cc:260] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.814388 1080 ts_tablet_manager.cc:1428] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:46.911212 1087 raft_consensus.cc:482] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:46.911309 1087 raft_consensus.cc:504] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.911600 1087 leader_election.cc:288] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041), 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919)
I20260810 06:35:46.916616 1090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.911989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:48695 (local address 127.30.71.1:44041)
0810 06:35:46.912043 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:46.912045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:46.912175 (+ 130us) server_negotiation.cc:407] Connection header received
0810 06:35:46.912269 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:46.912270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:46.912284 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:46.912345 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:46.912701 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.913490 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.915451 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.916044 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.916158 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.916178 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:46.916464 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:35:46.916546 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 06:35:46.916644 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.911847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:59921 (local address 127.30.71.2:43919)
0810 06:35:46.911991 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:46.911993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:46.912056 (+ 63us) server_negotiation.cc:407] Connection header received
0810 06:35:46.912204 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:46.912205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:46.912219 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:46.912320 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:46.912692 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.913482 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.915445 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.916171 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.916256 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.916268 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:46.916505 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 06:35:46.916563 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260810 06:35:46.916641 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.911798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.30.71.3:48695)
0810 06:35:46.912058 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:46.912068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:46.912124 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:46.912384 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:46.912385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:46.912390 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:46.912598 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.912600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.913595 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.913597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.915308 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.915312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.916226 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.916228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.916334 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.916340 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:35:46.916407 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:35:46.916461 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":85,"thread_start_us":160,"threads_started":1}
I20260810 06:35:46.916930 615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb7a3b64e334c6f97b376c7ff063196" candidate_uuid: "224257b2965242a58807c242ef04a1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b530c78fb948d696a93bbb2f8afcdc" is_pre_election: true
I20260810 06:35:46.917073 615 raft_consensus.cc:2454] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 224257b2965242a58807c242ef04a1cc in term 0.
I20260810 06:35:46.916646 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.911783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43919 (local address 127.30.71.3:59921)
0810 06:35:46.911976 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:46.911988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:46.912076 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:46.912384 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:46.912385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:46.912389 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:46.912529 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.912531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.913618 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.913618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.915344 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.915346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.916253 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.916254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.916347 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.916350 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:46.916439 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:35:46.916469 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":103,"thread_start_us":74,"threads_started":1}
I20260810 06:35:46.917313 844 leader_election.cc:302] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 224257b2965242a58807c242ef04a1cc, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:46.917466 1087 raft_consensus.cc:2781] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:46.917470 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb7a3b64e334c6f97b376c7ff063196" candidate_uuid: "224257b2965242a58807c242ef04a1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d" is_pre_election: true
I20260810 06:35:46.917537 1087 raft_consensus.cc:482] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:46.917563 1087 raft_consensus.cc:3037] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:46.917568 766 raft_consensus.cc:2454] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 224257b2965242a58807c242ef04a1cc in term 0.
I20260810 06:35:46.918390 1087 raft_consensus.cc:504] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.918548 1087 leader_election.cc:288] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [CANDIDATE]: Term 1 election: Requested vote from peers 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041), 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919)
I20260810 06:35:46.918720 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb7a3b64e334c6f97b376c7ff063196" candidate_uuid: "224257b2965242a58807c242ef04a1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47008aebce6a49ecbc21204b7f13903d"
I20260810 06:35:46.918809 766 raft_consensus.cc:3037] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:46.918748 615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eb7a3b64e334c6f97b376c7ff063196" candidate_uuid: "224257b2965242a58807c242ef04a1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b530c78fb948d696a93bbb2f8afcdc"
I20260810 06:35:46.918860 615 raft_consensus.cc:3037] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:46.919504 766 raft_consensus.cc:2454] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 224257b2965242a58807c242ef04a1cc in term 1.
I20260810 06:35:46.919507 615 raft_consensus.cc:2454] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 224257b2965242a58807c242ef04a1cc in term 1.
I20260810 06:35:46.919697 844 leader_election.cc:302] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 224257b2965242a58807c242ef04a1cc, 77b530c78fb948d696a93bbb2f8afcdc; no voters:
I20260810 06:35:46.919809 1087 raft_consensus.cc:2781] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:46.919970 1087 raft_consensus.cc:686] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 1 LEADER]: Becoming Leader. State: Replica: 224257b2965242a58807c242ef04a1cc, State: Running, Role: LEADER
I20260810 06:35:46.920027 1087 consensus_queue.cc:237] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } }
I20260810 06:35:46.921188 448 catalog_manager.cc:5495] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 224257b2965242a58807c242ef04a1cc (127.30.71.3). New cstate: current_term: 1 leader_uuid: "224257b2965242a58807c242ef04a1cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "224257b2965242a58807c242ef04a1cc" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46455 } health_report { overall_health: HEALTHY } } }
I20260810 06:35:46.936720 1104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.933844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50836 (local address 127.30.71.3:46455)
0810 06:35:46.934024 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:46.934026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:46.934038 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:35:46.934073 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:46.934074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:46.934093 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:46.934191 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:46.934307 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.934865 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.935727 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.935982 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.936075 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.936266 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:46.936388 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:46.936474 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:46.936533 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 06:35:46.936555 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260810 06:35:46.942660 1090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.939423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:51047 (local address 127.30.71.1:44041)
0810 06:35:46.939481 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:46.939483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:46.939591 (+ 108us) server_negotiation.cc:407] Connection header received
0810 06:35:46.939633 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:46.939634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:46.939646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:46.939697 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:46.939905 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.940454 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.942040 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.942230 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.942286 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.942462 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:46.942487 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:46.942525 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:46.942586 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 06:35:46.942605 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:35:46.942687 1108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.939340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.30.71.3:51047)
0810 06:35:46.939530 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:46.939540 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:46.939592 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:46.939710 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:46.939711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:46.939713 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:46.939787 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.939789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.940530 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.940531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.941950 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.941952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.942322 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.942323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.942371 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.942384 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:46.942544 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:46.942546 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:35:46.942584 (+ 38us) client_negotiation.cc:240] Negotiation successful
0810 06:35:46.942618 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260810 06:35:46.945212 615 raft_consensus.cc:1261] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Refusing update from remote peer 224257b2965242a58807c242ef04a1cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:46.945185 766 raft_consensus.cc:1261] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Refusing update from remote peer 224257b2965242a58807c242ef04a1cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:46.945494 1102 consensus_queue.cc:1035] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "77b530c78fb948d696a93bbb2f8afcdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:46.945634 1087 consensus_queue.cc:1035] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "47008aebce6a49ecbc21204b7f13903d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:46.953603 1104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.950972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:44029 (local address 127.30.71.3:46455)
0810 06:35:46.951053 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:46.951056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:46.951271 (+ 215us) server_negotiation.cc:407] Connection header received
0810 06:35:46.951320 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:46.951320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:46.951332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:46.951390 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:46.951479 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.951956 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.952816 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.953168 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.953240 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.953425 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:46.953451 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:46.953498 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:46.953537 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 06:35:46.953554 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:35:46.953615 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:46.950924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46455 (local address 127.30.71.1:44029)
0810 06:35:46.951178 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:46.951201 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:46.951262 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:46.951391 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:46.951392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:46.951393 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:35:46.951443 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.951444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.952016 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.952016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.952711 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:46.952713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:46.953252 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:46.953252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:46.953347 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:46.953352 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:35:46.953499 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:35:46.953500 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:35:46.953526 (+ 26us) client_negotiation.cc:240] Negotiation successful
0810 06:35:46.953561 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":175,"threads_started":1}
W20260810 06:35:46.954115 1051 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:35:48.478883 448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:48.479029 448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:48.480115 448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fa47fb4da4e42f2b17c435bb27416b3: Sending DeleteTablet for 3 replicas of tablet 1eb7a3b64e334c6f97b376c7ff063196
I20260810 06:35:48.480422 575 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb7a3b64e334c6f97b376c7ff063196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:48 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35384
I20260810 06:35:48.480501 745 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb7a3b64e334c6f97b376c7ff063196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:48 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:38532
I20260810 06:35:48.480726 1140 tablet_replica.cc:330] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc: stopping tablet replica
I20260810 06:35:48.480792 1141 tablet_replica.cc:330] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: stopping tablet replica
I20260810 06:35:48.480777 889 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb7a3b64e334c6f97b376c7ff063196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:48 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:35578
I20260810 06:35:48.480845 1140 raft_consensus.cc:2229] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:48.480878 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 524
I20260810 06:35:48.480909 1140 raft_consensus.cc:2258] T 1eb7a3b64e334c6f97b376c7ff063196 P 77b530c78fb948d696a93bbb2f8afcdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:48.480928 1141 raft_consensus.cc:2229] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:48.480970 1141 raft_consensus.cc:2258] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:48.482057 1142 tablet_replica.cc:330] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: stopping tablet replica
I20260810 06:35:48.482123 1142 raft_consensus.cc:2229] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:48.482234 1142 raft_consensus.cc:2258] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:48.482297 1141 ts_tablet_manager.cc:1905] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:48.482824 1142 ts_tablet_manager.cc:1905] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:48.484443 1141 ts_tablet_manager.cc:1918] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:35:48.484504 1141 log.cc:1196] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/1eb7a3b64e334c6f97b376c7ff063196
I20260810 06:35:48.484781 1141 ts_tablet_manager.cc:1939] T 1eb7a3b64e334c6f97b376c7ff063196 P 47008aebce6a49ecbc21204b7f13903d: Deleting consensus metadata
I20260810 06:35:48.484877 1142 ts_tablet_manager.cc:1918] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:35:48.484928 1142 log.cc:1196] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/1eb7a3b64e334c6f97b376c7ff063196
I20260810 06:35:48.485206 1142 ts_tablet_manager.cc:1939] T 1eb7a3b64e334c6f97b376c7ff063196 P 224257b2965242a58807c242ef04a1cc: Deleting consensus metadata
I20260810 06:35:48.485220 429 catalog_manager.cc:4841] TS 47008aebce6a49ecbc21204b7f13903d (127.30.71.2:43919): tablet 1eb7a3b64e334c6f97b376c7ff063196 (table test-table [id=74b127316f0b426fb36e3e021d93d741]) successfully deleted
I20260810 06:35:48.485818 432 catalog_manager.cc:4841] TS 224257b2965242a58807c242ef04a1cc (127.30.71.3:46455): tablet 1eb7a3b64e334c6f97b376c7ff063196 (table test-table [id=74b127316f0b426fb36e3e021d93d741]) successfully deleted
W20260810 06:35:48.490882 432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:44041 (error 108)
I20260810 06:35:48.491145 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 675
I20260810 06:35:48.492297 1144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:48.491319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44041 (local address 127.0.0.1:39766)
0810 06:35:48.492182 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:48.492245 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":63,"threads_started":1}
W20260810 06:35:48.492441 432 catalog_manager.cc:4584] TS 77b530c78fb948d696a93bbb2f8afcdc (127.30.71.1:44041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1eb7a3b64e334c6f97b376c7ff063196: Network error: Client connection negotiation failed: client connection to 127.30.71.1:44041: connect: Connection refused (error 111)
I20260810 06:35:48.499125 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 825
I20260810 06:35:48.506453 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 413
2026-08-10T06:35:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5848 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:48 dist-test-slave-w206 krb5kdc[1147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:48 dist-test-slave-w206 krb5kdc[1147](info): set up 2 sockets
Aug 10 06:35:48 dist-test-slave-w206 krb5kdc[1147](info): commencing operation
krb5kdc: starting...
W20260810 06:35:49.658694 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:49 dist-test-slave-w206 krb5kdc[1147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:49.789687 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:38925
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:49.873310 1162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:49.873546 1162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:49.873574 1162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:49.875061 1162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:49.875113 1162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:49.875128 1162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:49.875141 1162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:49.875221 1162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:49.876920 1162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:49.877199 1162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:49.877480 1162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:49.882951 1168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:49.883064 1162 server_base.cc:1034] running on GCE node
W20260810 06:35:49.882957 1167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:49.882957 1170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:49.883472 1162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:49.883752 1162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:49.884944 1162 hybrid_clock.cc:648] HybridClock initialized: now 1786343749884935 us; error 39 us; skew 500 ppm
Aug 10 06:35:49 dist-test-slave-w206 krb5kdc[1147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:49.891064 1162 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:49.891642 1162 webserver.cc:460] Webserver started at http://127.30.71.62:38645/ using document root <none> and password file <none>
I20260810 06:35:49.891927 1162 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:49.892002 1162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:49.892176 1162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:49.893587 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "e05428c2490c4fe58cf02c24c54b84a9"
format_stamp: "Formatted at 2026-08-10 06:35:49 on dist-test-slave-w206"
server_key: "011157f01d6a7d3b04a67c48b4afbc9d"
server_key_iv: "467b264ddc5d291dbc2899ff621f39e6"
server_key_version: "encryptionkey@0"
I20260810 06:35:49.894053 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "e05428c2490c4fe58cf02c24c54b84a9"
format_stamp: "Formatted at 2026-08-10 06:35:49 on dist-test-slave-w206"
server_key: "011157f01d6a7d3b04a67c48b4afbc9d"
server_key_iv: "467b264ddc5d291dbc2899ff621f39e6"
server_key_version: "encryptionkey@0"
I20260810 06:35:49.895576 1162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260810 06:35:49.896350 1177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:49.896544 1162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:49.896652 1162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "e05428c2490c4fe58cf02c24c54b84a9"
format_stamp: "Formatted at 2026-08-10 06:35:49 on dist-test-slave-w206"
server_key: "011157f01d6a7d3b04a67c48b4afbc9d"
server_key_iv: "467b264ddc5d291dbc2899ff621f39e6"
server_key_version: "encryptionkey@0"
I20260810 06:35:49.896727 1162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:49.911713 1162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:49.914134 1162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:49.914310 1162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:49.918588 1162 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:38925
I20260810 06:35:49.918625 1232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:38925 every 8 connection(s)
I20260810 06:35:49.919027 1162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:49.919924 1233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:49.922181 1233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9: Bootstrap starting.
I20260810 06:35:49.922730 1233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:49.923028 1233 log.cc:824] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:49.923719 1233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9: No bootstrap required, opened a new log
I20260810 06:35:49.925321 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 1162
I20260810 06:35:49.925298 1233 raft_consensus.cc:348] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05428c2490c4fe58cf02c24c54b84a9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38925 } }
I20260810 06:35:49.925438 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:49.925424 1233 raft_consensus.cc:374] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:49.925491 1233 raft_consensus.cc:729] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05428c2490c4fe58cf02c24c54b84a9, State: Initialized, Role: FOLLOWER
I20260810 06:35:49.925639 1233 consensus_queue.cc:260] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05428c2490c4fe58cf02c24c54b84a9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38925 } }
I20260810 06:35:49.925683 31004 external_mini_cluster.cc:1428] Setting key 2b3b7dda374057112e8c56629e8596b7
I20260810 06:35:49.925709 1233 raft_consensus.cc:388] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:49.925731 1233 raft_consensus.cc:482] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:49.925804 1233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:49.926879 1233 raft_consensus.cc:504] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05428c2490c4fe58cf02c24c54b84a9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38925 } }
I20260810 06:35:49.927026 1233 leader_election.cc:302] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e05428c2490c4fe58cf02c24c54b84a9; no voters:
I20260810 06:35:49.927217 1233 leader_election.cc:288] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:49.927287 1240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:49.927435 1240 raft_consensus.cc:686] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [term 1 LEADER]: Becoming Leader. State: Replica: e05428c2490c4fe58cf02c24c54b84a9, State: Running, Role: LEADER
I20260810 06:35:49.927490 1233 sys_catalog.cc:564] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:49.927523 1240 consensus_queue.cc:237] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05428c2490c4fe58cf02c24c54b84a9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38925 } }
I20260810 06:35:49.927896 1240 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e05428c2490c4fe58cf02c24c54b84a9. Latest consensus state: current_term: 1 leader_uuid: "e05428c2490c4fe58cf02c24c54b84a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05428c2490c4fe58cf02c24c54b84a9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38925 } } }
I20260810 06:35:49.927984 1240 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:49.927917 1241 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e05428c2490c4fe58cf02c24c54b84a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05428c2490c4fe58cf02c24c54b84a9" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38925 } } }
I20260810 06:35:49.928143 1241 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:49.928972 1249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:49.929548 1249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:49.931134 1249 catalog_manager.cc:1347] Generated new cluster ID: b178323e631a42819e20d80f8959805b
I20260810 06:35:49.931522 1249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:35:49 dist-test-slave-w206 krb5kdc[1147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:49.938620 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:49.926207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.30.71.62:38925)
0810 06:35:49.926416 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:49.926435 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:49.926462 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:35:49.926556 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:49.926561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:49.926603 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:49.926714 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:49.927624 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:49.929136 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:49.930164 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:49.930789 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:49.930923 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:49.933964 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:49.933968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:49.933968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:49.933980 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:49.936766 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:49.937129 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:49.937132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:49.937133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:49.937243 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:49.937416 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:49.937417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:49.937417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:49.937706 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:49.937845 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:49.938262 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:35:49.938391 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":124,"threads_started":1}
I20260810 06:35:49.957583 1249 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:49.958120 1249 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:49.960510 1249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e05428c2490c4fe58cf02c24c54b84a9: Generated new TSK 0
I20260810 06:35:49.960687 1249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:49.998683 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38925
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:50.094321 1264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:50.094571 1264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:50.094616 1264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:50.095835 1264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:50.095876 1264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:50.095902 1264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:50.097273 1264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:50.097457 1264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:50.097644 1264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:50.102835 1272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:50.102833 1269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:50.102829 1264 server_base.cc:1034] running on GCE node
W20260810 06:35:50.102861 1270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:50.103379 1264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:50.103713 1264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:50.104893 1264 hybrid_clock.cc:648] HybridClock initialized: now 1786343750104884 us; error 44 us; skew 500 ppm
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:50.107682 1264 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:50.108193 1264 webserver.cc:460] Webserver started at http://127.30.71.1:34765/ using document root <none> and password file <none>
I20260810 06:35:50.108456 1264 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:50.108528 1264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:50.108682 1264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:50.109748 1264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "0818a027db31471f942ca3c92e65a653"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "8fdd7d885b9710fc2c35bfba7555d5d9"
server_key_iv: "0f1d5867d644a48a0fd9b9b88c22ad1b"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.110139 1264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "0818a027db31471f942ca3c92e65a653"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "8fdd7d885b9710fc2c35bfba7555d5d9"
server_key_iv: "0f1d5867d644a48a0fd9b9b88c22ad1b"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.111610 1264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:50.112366 1279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.112538 1264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:50.112643 1264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "0818a027db31471f942ca3c92e65a653"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "8fdd7d885b9710fc2c35bfba7555d5d9"
server_key_iv: "0f1d5867d644a48a0fd9b9b88c22ad1b"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.112720 1264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:50.121172 1264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:50.123500 1264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:50.123665 1264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:50.123908 1264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:50.124213 1264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:50.124274 1264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.124316 1264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:50.124341 1264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.130786 1264 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:36855
I20260810 06:35:50.130864 1402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:36855 every 8 connection(s)
I20260810 06:35:50.131247 1264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:50.135267 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 1264
I20260810 06:35:50.135390 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:50.135581 31004 external_mini_cluster.cc:1428] Setting key a5f757a271bd3ad6061f95905f7ffff3
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:50.141690 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:50.132027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:39661 (local address 127.30.71.62:38925)
0810 06:35:50.132076 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:50.132078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:50.132342 (+ 264us) server_negotiation.cc:407] Connection header received
0810 06:35:50.132824 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:50.132825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:50.132834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:50.132897 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:50.133885 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.134412 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.135780 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.136039 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.136108 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:50.138441 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:50.138442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:50.138443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:50.138450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:50.140126 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:50.140661 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:50.140662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:50.140663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:50.140721 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:50.141013 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:50.141013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:50.141014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:50.141119 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:50.141232 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:50.141549 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 06:35:50.141620 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:35:50.142005 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:50.131993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:38925 (local address 127.30.71.1:39661)
0810 06:35:50.132174 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:50.132217 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:50.132738 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:50.132974 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:50.132978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:50.133554 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:50.133769 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:50.133773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.134531 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.134532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:50.135614 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:50.135617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.136136 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.136137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:50.136317 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:50.136893 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:50.136902 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:50.138333 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:50.140217 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:50.140225 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:50.140236 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:50.140581 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:50.140793 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:50.140794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:50.140795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:50.140944 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:50.141188 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:50.141195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:50.141392 (+ 197us) client_negotiation.cc:769] Sending connection context
0810 06:35:50.141519 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:35:50.141713 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260810 06:35:50.142596 1403 heartbeater.cc:347] Connected to a master server at 127.30.71.62:38925
I20260810 06:35:50.142697 1403 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:50.142925 1403 heartbeater.cc:511] Master 127.30.71.62:38925 requested a full tablet report, sending...
I20260810 06:35:50.143661 1196 ts_manager.cc:194] Registered new tserver with Master: 0818a027db31471f942ca3c92e65a653 (127.30.71.1:36855)
I20260810 06:35:50.144637 1196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:39661
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:50.164271 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38925
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:50.252221 1408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:50.252439 1408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:50.252465 1408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:50.253656 1408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:50.253703 1408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:50.253731 1408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:50.255312 1408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:50.255576 1408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:50.255823 1408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:50.261072 1413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:50.261078 1416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:50.261072 1414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:50.261437 1408 server_base.cc:1034] running on GCE node
I20260810 06:35:50.261677 1408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:50.261907 1408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:50.263048 1408 hybrid_clock.cc:648] HybridClock initialized: now 1786343750263034 us; error 30 us; skew 500 ppm
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:50.265733 1408 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:50.266201 1408 webserver.cc:460] Webserver started at http://127.30.71.2:34633/ using document root <none> and password file <none>
I20260810 06:35:50.266460 1408 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:50.266530 1408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:50.266671 1408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:50.267815 1408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "8d3e2d3ba871404e87af465e7dd57625"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "a66a1e81e00ce79be9e91a5ca71d190f"
server_key_iv: "51063e14c4f642137f53bcf24a3a9525"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.268172 1408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "8d3e2d3ba871404e87af465e7dd57625"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "a66a1e81e00ce79be9e91a5ca71d190f"
server_key_iv: "51063e14c4f642137f53bcf24a3a9525"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.269584 1408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:35:50.270354 1423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.270538 1408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:35:50.270643 1408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "8d3e2d3ba871404e87af465e7dd57625"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "a66a1e81e00ce79be9e91a5ca71d190f"
server_key_iv: "51063e14c4f642137f53bcf24a3a9525"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.270720 1408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:50.285171 1408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:50.287671 1408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:50.287860 1408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:50.288115 1408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:50.288453 1408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:50.288509 1408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.288551 1408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:50.288580 1408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.294639 1408 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:37761
I20260810 06:35:50.294724 1546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:37761 every 8 connection(s)
I20260810 06:35:50.295120 1408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:35:50.300757 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 1408
I20260810 06:35:50.300871 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:50.301111 31004 external_mini_cluster.cc:1428] Setting key 8c4034abca26cdb1c3c330768d373325
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:50.305547 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:50.295918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:57967 (local address 127.30.71.62:38925)
0810 06:35:50.295990 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:50.295993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:50.296173 (+ 180us) server_negotiation.cc:407] Connection header received
0810 06:35:50.296652 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:50.296653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:50.296664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:50.296747 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:50.297676 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.298173 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.299267 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.299461 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.299520 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:50.302368 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:50.302370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:50.302371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:50.302379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:50.303965 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:50.304492 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:50.304493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:50.304494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:50.304564 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:50.304816 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:50.304818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:50.304818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:50.304981 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:50.305056 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:50.305303 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 06:35:50.305405 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:35:50.305585 1549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:50.295872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:38925 (local address 127.30.71.2:57967)
0810 06:35:50.296042 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:50.296086 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:50.296566 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:50.296809 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:50.296813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:50.297386 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:50.297591 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:50.297595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.298260 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.298261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:50.299177 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:50.299180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.299537 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.299537 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:50.299662 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:50.300319 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:50.300328 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:50.302240 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:50.304055 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:50.304062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:50.304064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:50.304404 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:50.304634 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:50.304635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:50.304635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:50.304744 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:50.305051 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:50.305055 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:50.305172 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 06:35:50.305290 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:35:50.305421 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260810 06:35:50.306077 1547 heartbeater.cc:347] Connected to a master server at 127.30.71.62:38925
I20260810 06:35:50.306198 1547 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:50.306476 1547 heartbeater.cc:511] Master 127.30.71.62:38925 requested a full tablet report, sending...
I20260810 06:35:50.306878 1196 ts_manager.cc:194] Registered new tserver with Master: 8d3e2d3ba871404e87af465e7dd57625 (127.30.71.2:37761)
I20260810 06:35:50.307289 1196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:57967
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:50.330585 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38925
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:50.413259 1552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:50.413517 1552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:50.413555 1552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:50.414801 1552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:50.414862 1552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:50.414906 1552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:50.416589 1552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:50.416890 1552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:50.417131 1552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:50.422184 1557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:50.422211 1558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:50.422211 1560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:50.422415 1552 server_base.cc:1034] running on GCE node
I20260810 06:35:50.422668 1552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:50.422928 1552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:50.424131 1552 hybrid_clock.cc:648] HybridClock initialized: now 1786343750424111 us; error 38 us; skew 500 ppm
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:50.426750 1552 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:50.427332 1552 webserver.cc:460] Webserver started at http://127.30.71.3:33793/ using document root <none> and password file <none>
I20260810 06:35:50.427546 1552 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:50.427610 1552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:50.427740 1552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:50.428926 1552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "a40cb71c1fa443fba062c76351303124"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "839f7fea79195a6a3f7af32588ab269e"
server_key_iv: "a21dd3810193d7ddde8b6560c68a351d"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.429364 1552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "a40cb71c1fa443fba062c76351303124"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "839f7fea79195a6a3f7af32588ab269e"
server_key_iv: "a21dd3810193d7ddde8b6560c68a351d"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.430658 1552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:50.431416 1568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.431587 1552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.431687 1552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "a40cb71c1fa443fba062c76351303124"
format_stamp: "Formatted at 2026-08-10 06:35:50 on dist-test-slave-w206"
server_key: "839f7fea79195a6a3f7af32588ab269e"
server_key_iv: "a21dd3810193d7ddde8b6560c68a351d"
server_key_version: "encryptionkey@0"
I20260810 06:35:50.431763 1552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:50.453497 1552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:50.455792 1552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:50.455914 1552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:50.456171 1552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:50.456537 1552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:50.456575 1552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.456598 1552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:50.456611 1552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:50.462955 1552 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:40847
I20260810 06:35:50.463022 1691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:40847 every 8 connection(s)
I20260810 06:35:50.463359 1552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:35:50.466303 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 1552
I20260810 06:35:50.466404 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:50.466594 31004 external_mini_cluster.cc:1428] Setting key a9b555c0533370401550d90fa2810cb4
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:50.473380 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:50.464117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:57475 (local address 127.30.71.62:38925)
0810 06:35:50.464201 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:50.464204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:50.464433 (+ 229us) server_negotiation.cc:407] Connection header received
0810 06:35:50.464906 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:50.464907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:50.464919 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:50.464992 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:50.465925 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.466676 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.467985 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.468169 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.468239 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:50.470577 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:50.470578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:50.470579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:50.470585 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:50.471909 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:50.472345 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:50.472346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:50.472346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:50.472401 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:50.472782 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:50.472783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:50.472783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:50.472898 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:50.472954 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:50.473248 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 06:35:50.473314 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:35:50.473645 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:50.464071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:38925 (local address 127.30.71.3:57475)
0810 06:35:50.464303 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:50.464342 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:50.464820 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:50.465049 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:50.465052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:50.465638 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:50.465838 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:50.465842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.466811 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.466813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:50.467911 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:50.467914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:50.468286 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:50.468287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:50.468396 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:50.468921 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:50.468929 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:50.470458 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:50.471987 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:50.471993 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:50.471994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:50.472242 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:50.472521 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:50.472532 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:50.472532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:50.472700 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:50.472965 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:50.472971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:50.473099 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 06:35:50.473187 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:35:50.473331 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":163,"threads_started":1}
I20260810 06:35:50.474175 1692 heartbeater.cc:347] Connected to a master server at 127.30.71.62:38925
I20260810 06:35:50.474274 1692 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:50.474426 1692 heartbeater.cc:511] Master 127.30.71.62:38925 requested a full tablet report, sending...
I20260810 06:35:50.474848 1196 ts_manager.cc:194] Registered new tserver with Master: a40cb71c1fa443fba062c76351303124 (127.30.71.3:40847)
I20260810 06:35:50.475302 1196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:57475
I20260810 06:35:50.478783 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:35:50.493129 1239 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:35:50.491890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.30.71.62:38925)
0810 06:35:50.491987 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:50.491989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:50.492162 (+ 173us) server_negotiation.cc:407] Connection header received
0810 06:35:50.492281 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:50.492282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:50.492292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:50.492352 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:50.493059 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39392: BlockingRecv error: recv got EOF from 127.0.0.1:39392 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:35:50.494025 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 1264
I20260810 06:35:50.503528 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 1408
I20260810 06:35:50.510269 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 1552
I20260810 06:35:50.516338 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 1162
2026-08-10T06:35:50Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2007 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1705](info): set up 2 sockets
Aug 10 06:35:50 dist-test-slave-w206 krb5kdc[1705](info): commencing operation
krb5kdc: starting...
W20260810 06:35:51.642302 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:51 dist-test-slave-w206 krb5kdc[1705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:51.766536 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41327
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:41327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:51.848822 1718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:51.849089 1718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:51.849129 1718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:51.850327 1718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:51.850409 1718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:51.850436 1718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:51.850467 1718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:51.850492 1718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:51.852110 1718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:41327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:51.852393 1718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:51.852632 1718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:51.857729 1724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:51.857829 1718 server_base.cc:1034] running on GCE node
W20260810 06:35:51.857694 1723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:51.857721 1726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:51.858158 1718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:51.858480 1718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:51.859678 1718 hybrid_clock.cc:648] HybridClock initialized: now 1786343751859652 us; error 40 us; skew 500 ppm
Aug 10 06:35:51 dist-test-slave-w206 krb5kdc[1705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:51.862390 1718 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:51.862921 1718 webserver.cc:460] Webserver started at http://127.30.71.62:35851/ using document root <none> and password file <none>
I20260810 06:35:51.863217 1718 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:51.863294 1718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:51.863452 1718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:51.864557 1718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "c981e67fcd30427c809cfbf1ea41a5d1"
format_stamp: "Formatted at 2026-08-10 06:35:51 on dist-test-slave-w206"
server_key: "f51a1be72cec3883f86375e9e968f189"
server_key_iv: "c4468a9b83f2bec7d20ee90f6d3f2bdc"
server_key_version: "encryptionkey@0"
I20260810 06:35:51.864939 1718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "c981e67fcd30427c809cfbf1ea41a5d1"
format_stamp: "Formatted at 2026-08-10 06:35:51 on dist-test-slave-w206"
server_key: "f51a1be72cec3883f86375e9e968f189"
server_key_iv: "c4468a9b83f2bec7d20ee90f6d3f2bdc"
server_key_version: "encryptionkey@0"
I20260810 06:35:51.866335 1718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:35:51.867249 1734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:51.867479 1718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:51.867611 1718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "c981e67fcd30427c809cfbf1ea41a5d1"
format_stamp: "Formatted at 2026-08-10 06:35:51 on dist-test-slave-w206"
server_key: "f51a1be72cec3883f86375e9e968f189"
server_key_iv: "c4468a9b83f2bec7d20ee90f6d3f2bdc"
server_key_version: "encryptionkey@0"
I20260810 06:35:51.867697 1718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:51.892652 1718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:51.895432 1718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:51.895624 1718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:51.900056 1718 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:41327
I20260810 06:35:51.900095 1791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:41327 every 8 connection(s)
I20260810 06:35:51.900741 1718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:51.901437 1792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:51.902060 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 1718
I20260810 06:35:51.902153 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:51.902417 31004 external_mini_cluster.cc:1428] Setting key df3031cd06c612a9d2495fc3c342dba3
I20260810 06:35:51.904614 1792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1: Bootstrap starting.
I20260810 06:35:51.905370 1792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:51.905670 1792 log.cc:824] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:51.906447 1792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1: No bootstrap required, opened a new log
I20260810 06:35:51.908674 1792 raft_consensus.cc:348] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41327 } }
I20260810 06:35:51.908814 1792 raft_consensus.cc:374] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:51.908838 1792 raft_consensus.cc:729] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c981e67fcd30427c809cfbf1ea41a5d1, State: Initialized, Role: FOLLOWER
I20260810 06:35:51.908975 1792 consensus_queue.cc:260] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41327 } }
I20260810 06:35:51.909054 1792 raft_consensus.cc:388] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:51.909077 1792 raft_consensus.cc:482] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:51.909147 1792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:51.909912 1792 raft_consensus.cc:504] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41327 } }
Aug 10 06:35:51 dist-test-slave-w206 krb5kdc[1705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:51.910030 1792 leader_election.cc:302] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c981e67fcd30427c809cfbf1ea41a5d1; no voters:
I20260810 06:35:51.910225 1792 leader_election.cc:288] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:51.910284 1797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:51.910473 1797 raft_consensus.cc:686] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [term 1 LEADER]: Becoming Leader. State: Replica: c981e67fcd30427c809cfbf1ea41a5d1, State: Running, Role: LEADER
I20260810 06:35:51.910518 1792 sys_catalog.cc:564] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:51.910573 1797 consensus_queue.cc:237] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41327 } }
I20260810 06:35:51.911240 1798 sys_catalog.cc:455] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41327 } } }
I20260810 06:35:51.911348 1798 sys_catalog.cc:458] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:51.911687 1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c981e67fcd30427c809cfbf1ea41a5d1. Latest consensus state: current_term: 1 leader_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c981e67fcd30427c809cfbf1ea41a5d1" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41327 } } }
I20260810 06:35:51.911767 1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1 [sys.catalog]: This master's current role is: LEADER
W20260810 06:35:51.912451 1813 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:35:51.912526 1813 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:35:51.912585 1806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:51.912777 1806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:51.914491 1806 catalog_manager.cc:1347] Generated new cluster ID: 23d9c46553c14c57b01a56abbf848550
I20260810 06:35:51.914556 1806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:51.914638 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:51.903002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.30.71.62:41327)
0810 06:35:51.903218 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:51.903228 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:51.903263 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:35:51.903851 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:51.903860 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:51.903926 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:51.904083 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:51.904789 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:51.906169 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:51.907009 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:51.907629 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:51.907748 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:51.910559 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:51.910563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:51.910563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:51.910577 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:51.912850 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:51.913261 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:51.913265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:51.913265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:51.913368 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:51.913611 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:51.913612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:51.913612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:51.913929 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:51.914089 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:51.914229 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 06:35:51.914392 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":89,"threads_started":1}
I20260810 06:35:51.915643 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:35:51.920133 1806 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:51.920660 1806 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:51.927742 1806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c981e67fcd30427c809cfbf1ea41a5d1: Generated new TSK 0
I20260810 06:35:51.927944 1806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:35:51.940814 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:51.936410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.30.71.62:41327)
0810 06:35:51.936484 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:51.936487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:51.936649 (+ 162us) server_negotiation.cc:407] Connection header received
0810 06:35:51.936720 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:51.936721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:51.936732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:51.936792 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:51.937261 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:51.937850 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:51.939019 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:51.939319 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:51.939387 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:51.940205 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:51.940207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:51.940207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:51.940436 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:51.940454 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:51.940460 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:51.940517 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:51.940634 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 06:35:51.940666 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:35:51.943809 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 1718
2026-08-10T06:35:51Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1426 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:52 dist-test-slave-w206 krb5kdc[1826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:52 dist-test-slave-w206 krb5kdc[1826](info): set up 2 sockets
Aug 10 06:35:52 dist-test-slave-w206 krb5kdc[1826](info): commencing operation
krb5kdc: starting...
W20260810 06:35:53.029448 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:53.152238 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:38697
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:38697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:53.235208 1841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:53.235494 1841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:53.235532 1841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:53.236693 1841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:53.236775 1841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:53.236799 1841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:53.236819 1841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:53.236851 1841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:53.238350 1841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:38697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:38697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:53.238637 1841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:53.238873 1841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:53.243902 1849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:53.243964 1841 server_base.cc:1034] running on GCE node
W20260810 06:35:53.243927 1847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:53.243903 1846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:53.244379 1841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:53.244637 1841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:53.245783 1841 hybrid_clock.cc:648] HybridClock initialized: now 1786343753245762 us; error 37 us; skew 500 ppm
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:53.248528 1841 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:53.249023 1841 webserver.cc:460] Webserver started at http://127.30.71.62:37383/ using document root <none> and password file <none>
I20260810 06:35:53.249260 1841 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:53.249329 1841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:53.249485 1841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:53.250561 1841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "79dcea52674746cb98ddfff722ccfe92"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "b2fb5ba310e4da0da1a665520c3c1c00"
server_key_iv: "e77990947d931b0e2c8fb35b95fe9136"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.250939 1841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "79dcea52674746cb98ddfff722ccfe92"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "b2fb5ba310e4da0da1a665520c3c1c00"
server_key_iv: "e77990947d931b0e2c8fb35b95fe9136"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.252558 1841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:53.253358 1857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.253681 1841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:53.253782 1841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "79dcea52674746cb98ddfff722ccfe92"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "b2fb5ba310e4da0da1a665520c3c1c00"
server_key_iv: "e77990947d931b0e2c8fb35b95fe9136"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.253861 1841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:53.266286 1841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:53.268843 1841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:53.269022 1841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:53.273520 1841 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:38697
I20260810 06:35:53.273594 1914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:38697 every 8 connection(s)
I20260810 06:35:53.273952 1841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:53.275231 1915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:53.277648 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 1841
I20260810 06:35:53.277673 1915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92: Bootstrap starting.
I20260810 06:35:53.277776 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:53.277969 31004 external_mini_cluster.cc:1428] Setting key 98d171893acef0278b8c4f782616362a
I20260810 06:35:53.278416 1915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:53.278766 1915 log.cc:824] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:53.279592 1915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92: No bootstrap required, opened a new log
I20260810 06:35:53.281857 1915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dcea52674746cb98ddfff722ccfe92" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38697 } }
I20260810 06:35:53.282035 1915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:53.282073 1915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79dcea52674746cb98ddfff722ccfe92, State: Initialized, Role: FOLLOWER
I20260810 06:35:53.282196 1915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dcea52674746cb98ddfff722ccfe92" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38697 } }
I20260810 06:35:53.282297 1915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:53.282328 1915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:53.282371 1915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:53.283097 1915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dcea52674746cb98ddfff722ccfe92" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38697 } }
I20260810 06:35:53.283192 1915 leader_election.cc:302] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79dcea52674746cb98ddfff722ccfe92; no voters:
I20260810 06:35:53.283419 1915 leader_election.cc:288] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:53.283512 1920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:53.283732 1915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:53.284036 1920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [term 1 LEADER]: Becoming Leader. State: Replica: 79dcea52674746cb98ddfff722ccfe92, State: Running, Role: LEADER
I20260810 06:35:53.284144 1920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dcea52674746cb98ddfff722ccfe92" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38697 } }
I20260810 06:35:53.284520 1920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79dcea52674746cb98ddfff722ccfe92. Latest consensus state: current_term: 1 leader_uuid: "79dcea52674746cb98ddfff722ccfe92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dcea52674746cb98ddfff722ccfe92" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38697 } } }
I20260810 06:35:53.284616 1920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:53.284528 1921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79dcea52674746cb98ddfff722ccfe92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79dcea52674746cb98ddfff722ccfe92" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38697 } } }
I20260810 06:35:53.284709 1921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:53.284845 1930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:53.285362 1930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:53.287226 1930 catalog_manager.cc:1347] Generated new cluster ID: 42b0d57e3b304f8eb14fbbf915218f0a
I20260810 06:35:53.287292 1930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:53.290186 1918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.278555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41198 (local address 127.30.71.62:38697)
0810 06:35:53.278789 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:53.278800 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:53.278846 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:35:53.279557 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:53.279565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:53.279620 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:53.279800 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:53.280645 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.281674 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.282755 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.283284 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.283461 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.286547 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:53.286551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:53.286551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:53.286563 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:53.288523 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.288973 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.288977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.288977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.289092 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.289313 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.289314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.289315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.289567 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:53.289751 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:53.289859 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:35:53.290004 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
I20260810 06:35:53.297649 1930 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:53.298484 1930 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:53.310849 1930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79dcea52674746cb98ddfff722ccfe92: Generated new TSK 0
I20260810 06:35:53.311120 1930 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:53.342733 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38697
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:53.427429 1942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:53.427649 1942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:53.427673 1942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:53.428892 1942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:53.428941 1942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:53.428968 1942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:53.430486 1942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:53.430752 1942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:53.431049 1942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:53.436491 1948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:53.436502 1950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:53.436496 1947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:53.436597 1942 server_base.cc:1034] running on GCE node
I20260810 06:35:53.436932 1942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:53.437247 1942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:53.438422 1942 hybrid_clock.cc:648] HybridClock initialized: now 1786343753438400 us; error 40 us; skew 500 ppm
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:53.441044 1942 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:53.441573 1942 webserver.cc:460] Webserver started at http://127.30.71.1:36107/ using document root <none> and password file <none>
I20260810 06:35:53.441810 1942 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:53.441880 1942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:53.442044 1942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:53.443202 1942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "d169835bd8b348e893c0b7e049aab233"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "862ce043bfe4ff51d615498c63c9a92a"
server_key_iv: "f0adda795c878e4bb61bd7a3c83a5270"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.443579 1942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "d169835bd8b348e893c0b7e049aab233"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "862ce043bfe4ff51d615498c63c9a92a"
server_key_iv: "f0adda795c878e4bb61bd7a3c83a5270"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.444919 1942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:35:53.445678 1957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.445854 1942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:53.445959 1942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "d169835bd8b348e893c0b7e049aab233"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "862ce043bfe4ff51d615498c63c9a92a"
server_key_iv: "f0adda795c878e4bb61bd7a3c83a5270"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.446044 1942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:53.462040 1942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:53.464134 1942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:53.464310 1942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:53.464598 1942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:53.464949 1942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:53.465008 1942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.465044 1942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:53.465091 1942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.471273 1942 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:41607
I20260810 06:35:53.471328 2078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:41607 every 8 connection(s)
I20260810 06:35:53.471652 1942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:53.478747 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 1942
I20260810 06:35:53.478868 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:53.479117 31004 external_mini_cluster.cc:1428] Setting key ac06ca6995ced57bfc3f63a649e38300
I20260810 06:35:53.482043 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.472391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:38697 (local address 127.30.71.1:45029)
0810 06:35:53.472561 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:53.472605 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:53.473112 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:53.473402 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:53.473405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:53.474017 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:53.474213 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:53.474218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.474897 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.474897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:53.475789 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:53.475792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.476167 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.476168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:53.476273 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.476750 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:53.476758 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:53.478294 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:53.480328 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:53.480360 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:53.480363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:53.480713 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:53.480950 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:53.480951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:53.480952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:53.481108 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:53.481367 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:53.481374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:53.481559 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 06:35:53.481694 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 06:35:53.481856 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260810 06:35:53.482075 1918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.472436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:45029 (local address 127.30.71.62:38697)
0810 06:35:53.472510 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:53.472512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:53.472697 (+ 185us) server_negotiation.cc:407] Connection header received
0810 06:35:53.473238 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:53.473240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:53.473252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:53.473319 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:53.474311 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.474815 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.475883 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.476082 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.476131 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.478401 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:53.478403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:53.478403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:53.478410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:53.480199 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.480816 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.480817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.480817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.480876 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.481176 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.481177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.481177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.481298 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:53.481353 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:53.481831 (+ 478us) server_negotiation.cc:299] Negotiation successful
0810 06:35:53.481939 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:35:53.482740 2080 heartbeater.cc:347] Connected to a master server at 127.30.71.62:38697
I20260810 06:35:53.482861 2080 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:53.483116 2080 heartbeater.cc:511] Master 127.30.71.62:38697 requested a full tablet report, sending...
I20260810 06:35:53.483765 1876 ts_manager.cc:194] Registered new tserver with Master: d169835bd8b348e893c0b7e049aab233 (127.30.71.1:41607)
I20260810 06:35:53.484560 1876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:45029
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:53.509694 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38697
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:53.593940 2086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:53.594158 2086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:53.594183 2086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:53.595443 2086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:53.595487 2086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:53.595515 2086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:53.596946 2086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:53.597136 2086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:53.597342 2086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:53.603008 2091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:53.603178 2086 server_base.cc:1034] running on GCE node
W20260810 06:35:53.603050 2092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:53.603008 2094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:53.603473 2086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:53.603688 2086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:53.604847 2086 hybrid_clock.cc:648] HybridClock initialized: now 1786343753604825 us; error 38 us; skew 500 ppm
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:53.607501 2086 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:53.608006 2086 webserver.cc:460] Webserver started at http://127.30.71.2:36273/ using document root <none> and password file <none>
I20260810 06:35:53.608273 2086 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:53.608347 2086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:53.608502 2086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:53.609612 2086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "4ae742317dc341a5884266c2f588c441"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "61db23165e36d98e902db4a5cb1721f7"
server_key_iv: "c6466ac83a7be8ed621fd01c0fb63989"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.609990 2086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "4ae742317dc341a5884266c2f588c441"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "61db23165e36d98e902db4a5cb1721f7"
server_key_iv: "c6466ac83a7be8ed621fd01c0fb63989"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.611433 2086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:53.612193 2101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.612376 2086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:53.612468 2086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "4ae742317dc341a5884266c2f588c441"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "61db23165e36d98e902db4a5cb1721f7"
server_key_iv: "c6466ac83a7be8ed621fd01c0fb63989"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.612553 2086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:53.628737 2086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:53.630760 2086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:53.630946 2086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:53.631309 2086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:53.631644 2086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:53.631708 2086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.631744 2086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:53.631776 2086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.637864 2086 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:40191
I20260810 06:35:53.637912 2222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:40191 every 8 connection(s)
I20260810 06:35:53.638242 2086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:53.645618 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 2086
I20260810 06:35:53.645732 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:53.645941 31004 external_mini_cluster.cc:1428] Setting key 4bf1093c741cf3a4ba079e8fe13d0bdd
I20260810 06:35:53.648047 1918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.639000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:55295 (local address 127.30.71.62:38697)
0810 06:35:53.639079 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:53.639082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:53.639306 (+ 224us) server_negotiation.cc:407] Connection header received
0810 06:35:53.639779 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:53.639781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:53.639791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:53.639844 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:53.640872 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.641364 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.642373 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.642571 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.642621 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.644867 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:53.644868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:53.644868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:53.644876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:53.646446 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.646903 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.646905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.646905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.646961 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.647344 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.647346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.647346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.647475 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:53.647549 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:53.647907 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 06:35:53.647983 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:35:53.648326 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.638935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:38697 (local address 127.30.71.2:55295)
0810 06:35:53.639164 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:53.639220 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:53.639702 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:53.639911 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:53.639915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:53.640589 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:53.640791 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:53.640798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.641446 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.641447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:53.642300 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:53.642303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.642642 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.642643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:53.642743 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.643230 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:53.643238 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:53.644772 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:53.646514 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:53.646521 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:53.646523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:53.646822 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:53.647080 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:53.647080 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:53.647081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:53.647229 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:53.647554 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:53.647560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:53.647744 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 06:35:53.647862 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:35:53.648050 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":148,"threads_started":1}
I20260810 06:35:53.649003 2223 heartbeater.cc:347] Connected to a master server at 127.30.71.62:38697
I20260810 06:35:53.649119 2223 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:53.649348 2223 heartbeater.cc:511] Master 127.30.71.62:38697 requested a full tablet report, sending...
I20260810 06:35:53.649801 1876 ts_manager.cc:194] Registered new tserver with Master: 4ae742317dc341a5884266c2f588c441 (127.30.71.2:40191)
I20260810 06:35:53.650187 1876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:55295
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:53.674176 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38697
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:53.757346 2228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:53.757606 2228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:53.757647 2228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:53.758858 2228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:53.758929 2228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:53.758977 2228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:53.760530 2228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:53.760814 2228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:53.761051 2228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:53.766144 2236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:53.766144 2233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:53.766183 2234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:53.766592 2228 server_base.cc:1034] running on GCE node
I20260810 06:35:53.766800 2228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:53.767060 2228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:53.768219 2228 hybrid_clock.cc:648] HybridClock initialized: now 1786343753768207 us; error 29 us; skew 500 ppm
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:53.771109 2228 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:53.771585 2228 webserver.cc:460] Webserver started at http://127.30.71.3:33151/ using document root <none> and password file <none>
I20260810 06:35:53.771816 2228 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:53.771863 2228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:53.772004 2228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:53.773079 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "b192eaa1ccd54d07b1e5c5e63f000258"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "8e479702be50e4689d5d25ee76faf663"
server_key_iv: "58c6adffc47b6417a4f1f799e7a07485"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.773463 2228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "b192eaa1ccd54d07b1e5c5e63f000258"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "8e479702be50e4689d5d25ee76faf663"
server_key_iv: "58c6adffc47b6417a4f1f799e7a07485"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.774863 2228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:35:53.775728 2243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.775900 2228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:53.776007 2228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "b192eaa1ccd54d07b1e5c5e63f000258"
format_stamp: "Formatted at 2026-08-10 06:35:53 on dist-test-slave-w206"
server_key: "8e479702be50e4689d5d25ee76faf663"
server_key_iv: "58c6adffc47b6417a4f1f799e7a07485"
server_key_version: "encryptionkey@0"
I20260810 06:35:53.776084 2228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:53.790957 2228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:53.793576 2228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:53.793751 2228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:53.794020 2228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:53.794399 2228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:53.794457 2228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.794502 2228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:53.794526 2228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:53.800755 2228 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:37061
I20260810 06:35:53.800830 2361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:37061 every 8 connection(s)
I20260810 06:35:53.801185 2228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:53.810016 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 2228
I20260810 06:35:53.810127 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:53.810364 31004 external_mini_cluster.cc:1428] Setting key a46dbd28947ace42b7770fc45cd0dc49
I20260810 06:35:53.810483 1918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.802005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:37705 (local address 127.30.71.62:38697)
0810 06:35:53.802080 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:53.802082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:53.802277 (+ 195us) server_negotiation.cc:407] Connection header received
0810 06:35:53.802778 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:53.802779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:53.802789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:53.802853 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:53.803909 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.804394 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.805364 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.805556 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.805611 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.807811 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:53.807812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:53.807812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:53.807819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:53.809054 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.809426 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.809426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.809427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.809477 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.809685 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.809685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.809686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.809796 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:53.809850 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:53.810261 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 06:35:53.810358 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:35:53.810613 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.801953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:38697 (local address 127.30.71.3:37705)
0810 06:35:53.802142 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:53.802184 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:53.802684 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:53.802932 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:53.802936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:53.803619 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:53.803820 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:53.803825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.804478 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.804478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:53.805272 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:53.805275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.805632 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.805633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:53.805737 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.806236 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:53.806244 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:53.807725 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:53.809129 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:53.809135 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:53.809137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:53.809362 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:53.809534 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:53.809535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:53.809535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:53.809633 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:53.809871 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:53.809877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:53.810052 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 06:35:53.810190 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 06:35:53.810371 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260810 06:35:53.811108 2362 heartbeater.cc:347] Connected to a master server at 127.30.71.62:38697
I20260810 06:35:53.811249 2362 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:53.811493 2362 heartbeater.cc:511] Master 127.30.71.62:38697 requested a full tablet report, sending...
I20260810 06:35:53.811882 1876 ts_manager.cc:194] Registered new tserver with Master: b192eaa1ccd54d07b1e5c5e63f000258 (127.30.71.3:37061)
I20260810 06:35:53.812290 1876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:37705
I20260810 06:35:53.822633 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[1826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343753, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:53.847841 1918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:53.839746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.30.71.62:38697)
0810 06:35:53.839820 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:53.839822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:53.839969 (+ 147us) server_negotiation.cc:407] Connection header received
0810 06:35:53.840031 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:53.840032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:53.840044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:53.840097 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:53.840790 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.841270 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.842073 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:53.842278 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:53.842345 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:53.844870 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:53.844872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:53.844872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:53.844879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:53.846292 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.846649 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.846650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.846650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.846713 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:53.846957 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:53.846958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:53.846958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:53.847381 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:53.847466 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:53.847653 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 06:35:53.847720 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 06:35:53.848093 1876 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41204
I20260810 06:35:53.848706 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 1942
I20260810 06:35:53.855837 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 2086
I20260810 06:35:53.863126 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 2228
I20260810 06:35:53.868506 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 1841
2026-08-10T06:35:53Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1925 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[2374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[2374](info): set up 2 sockets
Aug 10 06:35:53 dist-test-slave-w206 krb5kdc[2374](info): commencing operation
krb5kdc: starting...
W20260810 06:35:55.014111 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:55.150951 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43619
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:43619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:55.235460 2387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:55.235689 2387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:55.235713 2387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:55.236900 2387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:55.236945 2387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:55.236959 2387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:55.236972 2387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:55.236985 2387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:55.238531 2387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:43619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:55.238792 2387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:55.239075 2387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:55.244095 2393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:55.244196 2387 server_base.cc:1034] running on GCE node
W20260810 06:35:55.244089 2392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:55.244089 2395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:55.244565 2387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:55.244834 2387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:55.245986 2387 hybrid_clock.cc:648] HybridClock initialized: now 1786343755245949 us; error 50 us; skew 500 ppm
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:55.248643 2387 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:55.249110 2387 webserver.cc:460] Webserver started at http://127.30.71.62:39113/ using document root <none> and password file <none>
I20260810 06:35:55.249352 2387 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:55.249398 2387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:55.249555 2387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:55.250660 2387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "232df739b6d046ab8ac7f05bd0c4da14"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "8424598a1ebc87e3b3986d96cb509f04"
server_key_iv: "2aff4903f859beb702ecab581663760a"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.251092 2387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "232df739b6d046ab8ac7f05bd0c4da14"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "8424598a1ebc87e3b3986d96cb509f04"
server_key_iv: "2aff4903f859beb702ecab581663760a"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.252559 2387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:55.253353 2402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.253547 2387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:55.253648 2387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "232df739b6d046ab8ac7f05bd0c4da14"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "8424598a1ebc87e3b3986d96cb509f04"
server_key_iv: "2aff4903f859beb702ecab581663760a"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.253736 2387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:55.273178 2387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:55.275717 2387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:55.275908 2387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:55.280053 2387 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:43619
I20260810 06:35:55.280089 2460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:43619 every 8 connection(s)
I20260810 06:35:55.280593 2387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:55.281284 2461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:55.283779 2461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14: Bootstrap starting.
I20260810 06:35:55.284399 2461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:55.284715 2461 log.cc:824] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:55.285509 2461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14: No bootstrap required, opened a new log
I20260810 06:35:55.287240 2461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232df739b6d046ab8ac7f05bd0c4da14" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43619 } }
I20260810 06:35:55.287384 2461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:55.287407 2461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232df739b6d046ab8ac7f05bd0c4da14, State: Initialized, Role: FOLLOWER
I20260810 06:35:55.287554 2461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232df739b6d046ab8ac7f05bd0c4da14" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43619 } }
I20260810 06:35:55.287658 2461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:55.287700 2461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:55.287766 2461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:55.287829 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 2387
I20260810 06:35:55.287942 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:55.288136 31004 external_mini_cluster.cc:1428] Setting key ae0e73a03496adc999b247bce17ab52e
I20260810 06:35:55.289086 2461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232df739b6d046ab8ac7f05bd0c4da14" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43619 } }
I20260810 06:35:55.289211 2461 leader_election.cc:302] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 232df739b6d046ab8ac7f05bd0c4da14; no voters:
I20260810 06:35:55.289383 2461 leader_election.cc:288] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:55.289440 2466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:55.289595 2466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [term 1 LEADER]: Becoming Leader. State: Replica: 232df739b6d046ab8ac7f05bd0c4da14, State: Running, Role: LEADER
I20260810 06:35:55.289659 2461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:55.289702 2466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232df739b6d046ab8ac7f05bd0c4da14" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43619 } }
I20260810 06:35:55.290652 2467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 232df739b6d046ab8ac7f05bd0c4da14. Latest consensus state: current_term: 1 leader_uuid: "232df739b6d046ab8ac7f05bd0c4da14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232df739b6d046ab8ac7f05bd0c4da14" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43619 } } }
I20260810 06:35:55.290776 2467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:55.290866 2468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "232df739b6d046ab8ac7f05bd0c4da14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232df739b6d046ab8ac7f05bd0c4da14" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43619 } } }
I20260810 06:35:55.290957 2468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:55.291088 2475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:55.291654 2475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:55.293305 2475 catalog_manager.cc:1347] Generated new cluster ID: bafbece04a864858b51bbd4ad46d9c50
I20260810 06:35:55.293367 2475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:55.301792 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.288666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.30.71.62:43619)
0810 06:35:55.288863 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:55.288872 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:55.288909 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:35:55.289013 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:55.289022 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:55.289065 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:55.289226 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:55.290018 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.291681 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.292800 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.293301 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.293413 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.297804 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:55.297809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:55.297809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:55.297823 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:55.300009 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.300457 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.300460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.300461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.300574 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.300825 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.300826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.300826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.301132 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:55.301294 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:55.301415 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:35:55.301557 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260810 06:35:55.308003 2475 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:55.308739 2475 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:55.315436 2475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 232df739b6d046ab8ac7f05bd0c4da14: Generated new TSK 0
I20260810 06:35:55.315646 2475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:55.346275 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43619
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:55.430032 2489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:55.430284 2489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:55.430325 2489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:55.431540 2489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:55.431609 2489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:55.431655 2489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:55.433173 2489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:55.433445 2489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:55.433684 2489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:55.438686 2496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:55.438802 2498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:55.438740 2495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:55.438867 2489 server_base.cc:1034] running on GCE node
I20260810 06:35:55.439150 2489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:55.439445 2489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:55.440598 2489 hybrid_clock.cc:648] HybridClock initialized: now 1786343755440585 us; error 45 us; skew 500 ppm
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:55.443312 2489 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:55.443796 2489 webserver.cc:460] Webserver started at http://127.30.71.1:33047/ using document root <none> and password file <none>
I20260810 06:35:55.444024 2489 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:55.444094 2489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:55.444254 2489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:55.445314 2489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "9f1ac61d1dfc4320907dd5a034a1b870"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "b875a0f82e751c466539e11418469747"
server_key_iv: "f48bcbbffba07a22b32c49b7c5a29d33"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.445685 2489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "9f1ac61d1dfc4320907dd5a034a1b870"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "b875a0f82e751c466539e11418469747"
server_key_iv: "f48bcbbffba07a22b32c49b7c5a29d33"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.447106 2489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:35:55.447891 2505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.448071 2489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:55.448172 2489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "9f1ac61d1dfc4320907dd5a034a1b870"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "b875a0f82e751c466539e11418469747"
server_key_iv: "f48bcbbffba07a22b32c49b7c5a29d33"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.448267 2489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:55.461119 2489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:55.463357 2489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:55.463533 2489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:55.463806 2489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:55.464136 2489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:55.464192 2489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.464241 2489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:55.464273 2489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.470332 2489 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:40163
I20260810 06:35:55.470379 2628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:40163 every 8 connection(s)
I20260810 06:35:55.470721 2489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:55.472175 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 2489
I20260810 06:35:55.472337 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:55.472551 31004 external_mini_cluster.cc:1428] Setting key 925f8ad2045f366c4f13cb3e326cbd6d
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:55.483150 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.471586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:55631 (local address 127.30.71.62:43619)
0810 06:35:55.471632 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:55.471634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:55.471863 (+ 229us) server_negotiation.cc:407] Connection header received
0810 06:35:55.473041 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:55.473042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:55.473059 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:55.473124 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:55.473914 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.474708 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.476584 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.476814 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.476899 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.479872 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:55.479874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:55.479874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:55.479880 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:55.481414 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.481991 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.481992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.481992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.482045 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.482363 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.482363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.482364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.482470 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:55.482524 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:55.482942 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 06:35:55.483079 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:35:55.483435 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.471536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43619 (local address 127.30.71.1:55631)
0810 06:35:55.471703 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:55.471747 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:55.472920 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:55.473223 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:55.473228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:55.473539 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:55.473740 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:55.473744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.474823 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.474824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:55.476475 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:55.476479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.476929 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.476931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:55.477100 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.477688 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:55.477708 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:55.479761 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:55.481516 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:55.481524 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:55.481527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:55.481906 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:55.482119 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:55.482120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:55.482120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:55.482285 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:55.482548 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:55.482556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:55.482770 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 06:35:55.482909 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 06:35:55.483160 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:35:55.484035 2629 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43619
I20260810 06:35:55.484128 2629 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:55.484326 2629 heartbeater.cc:511] Master 127.30.71.62:43619 requested a full tablet report, sending...
I20260810 06:35:55.485211 2423 ts_manager.cc:194] Registered new tserver with Master: 9f1ac61d1dfc4320907dd5a034a1b870 (127.30.71.1:40163)
I20260810 06:35:55.485857 2423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:55631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:55.504371 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43619
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:55.597497 2634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:55.597726 2634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:55.597754 2634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:55.598966 2634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:55.599023 2634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:55.599050 2634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:55.600620 2634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:55.600821 2634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:55.601004 2634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:55.606343 2639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:55.606374 2642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:55.606343 2640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:55.606743 2634 server_base.cc:1034] running on GCE node
I20260810 06:35:55.607060 2634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:55.607328 2634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:55.608472 2634 hybrid_clock.cc:648] HybridClock initialized: now 1786343755608453 us; error 39 us; skew 500 ppm
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:55.611327 2634 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:55.611840 2634 webserver.cc:460] Webserver started at http://127.30.71.2:46345/ using document root <none> and password file <none>
I20260810 06:35:55.612100 2634 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:55.612172 2634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:55.612341 2634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:55.613430 2634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "617d819c8be34f659e3795a43a801bda"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "f7fad50ac97e09ddd951eccfb09f096f"
server_key_iv: "db2e047a53d19091d89d6fd9bcdec8ee"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.613807 2634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "617d819c8be34f659e3795a43a801bda"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "f7fad50ac97e09ddd951eccfb09f096f"
server_key_iv: "db2e047a53d19091d89d6fd9bcdec8ee"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.615226 2634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:35:55.615978 2649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.616147 2634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:55.616238 2634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "617d819c8be34f659e3795a43a801bda"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "f7fad50ac97e09ddd951eccfb09f096f"
server_key_iv: "db2e047a53d19091d89d6fd9bcdec8ee"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.616329 2634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:55.631953 2634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:55.634472 2634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:55.634652 2634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:55.634927 2634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:55.635432 2634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:55.635499 2634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.635545 2634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:55.635571 2634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.641911 2634 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:34669
I20260810 06:35:55.641955 2769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:34669 every 8 connection(s)
I20260810 06:35:55.642350 2634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:55.650813 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 2634
I20260810 06:35:55.650995 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:55.651228 31004 external_mini_cluster.cc:1428] Setting key ddd0ff20e35423f7f37bc6e59ab52345
I20260810 06:35:55.652846 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.643179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:54609 (local address 127.30.71.62:43619)
0810 06:35:55.643252 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:55.643254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:55.643534 (+ 280us) server_negotiation.cc:407] Connection header received
0810 06:35:55.644570 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:55.644571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:55.644583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:55.644639 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:55.645252 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.645778 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.646912 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.647174 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.647232 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.649556 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:55.649557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:55.649557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:55.649564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:55.651169 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.651737 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.651738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.651738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.651805 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.652084 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.652085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.652085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.652242 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:55.652318 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:55.652601 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:35:55.652701 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:35:55.652880 2772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.643167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43619 (local address 127.30.71.2:54609)
0810 06:35:55.643356 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:55.643422 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:55.644470 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:55.644697 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:55.644700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:55.644980 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:55.645177 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:55.645181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.645892 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.645893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:55.646823 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:55.646826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.647274 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.647275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:55.647409 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.647909 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:55.647918 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:55.649453 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:55.651312 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:55.651323 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:55.651325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:55.651655 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:55.651880 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:55.651881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:55.651882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:55.652034 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:55.652318 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:55.652330 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:55.652463 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 06:35:55.652554 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:35:55.652684 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260810 06:35:55.654000 2770 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43619
I20260810 06:35:55.654129 2770 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:55.654315 2770 heartbeater.cc:511] Master 127.30.71.62:43619 requested a full tablet report, sending...
I20260810 06:35:55.654788 2423 ts_manager.cc:194] Registered new tserver with Master: 617d819c8be34f659e3795a43a801bda (127.30.71.2:34669)
I20260810 06:35:55.655355 2423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:54609
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:55.680054 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43619
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:55.764556 2776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:55.764775 2776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:55.764801 2776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:55.766010 2776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:55.766054 2776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:55.766080 2776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:55.767622 2776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:55.767817 2776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:55.767993 2776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:55.773101 2782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:55.773108 2783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:55.773229 2786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:55.773407 2776 server_base.cc:1034] running on GCE node
I20260810 06:35:55.773671 2776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:55.773933 2776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:55.775068 2776 hybrid_clock.cc:648] HybridClock initialized: now 1786343755775045 us; error 27 us; skew 500 ppm
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:55.777735 2776 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:55.778239 2776 webserver.cc:460] Webserver started at http://127.30.71.3:38499/ using document root <none> and password file <none>
I20260810 06:35:55.778473 2776 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:55.778543 2776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:55.778702 2776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:55.779841 2776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "18c6fdc6a2a14772ba5f3fe70d3c67f4"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "6d95cc7aa8f882fcbb2c7931deedc941"
server_key_iv: "beebfde3a9dd2f9f12a191862bfd9684"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.780241 2776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "18c6fdc6a2a14772ba5f3fe70d3c67f4"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "6d95cc7aa8f882fcbb2c7931deedc941"
server_key_iv: "beebfde3a9dd2f9f12a191862bfd9684"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.781591 2776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:35:55.782390 2793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.782562 2776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:55.782667 2776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "18c6fdc6a2a14772ba5f3fe70d3c67f4"
format_stamp: "Formatted at 2026-08-10 06:35:55 on dist-test-slave-w206"
server_key: "6d95cc7aa8f882fcbb2c7931deedc941"
server_key_iv: "beebfde3a9dd2f9f12a191862bfd9684"
server_key_version: "encryptionkey@0"
I20260810 06:35:55.782744 2776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:55.795202 2776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:55.797585 2776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:55.797751 2776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:55.798002 2776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:55.798329 2776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:55.798384 2776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.798427 2776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:55.798453 2776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:55.804693 2776 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:38137
I20260810 06:35:55.804741 2915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:38137 every 8 connection(s)
I20260810 06:35:55.805091 2776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:35:55.805789 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 2776
I20260810 06:35:55.805891 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:55.806126 31004 external_mini_cluster.cc:1428] Setting key 47bfe65082d2a8d69106531bf4c7e36b
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:55.815002 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.806278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:51273 (local address 127.30.71.62:43619)
0810 06:35:55.806330 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:55.806332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:55.806343 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:35:55.807509 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:55.807511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:55.807524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:55.807570 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:55.808234 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.808735 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.809781 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.809986 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.810075 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.812372 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:55.812374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:55.812374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:55.812381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:55.813553 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.813962 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.813963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.813964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.814030 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.814294 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.814296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.814296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.814415 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:55.814497 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:55.814789 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 06:35:55.814875 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:35:55.815165 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.805973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43619 (local address 127.30.71.3:51273)
0810 06:35:55.806147 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:55.806207 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:55.807397 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:55.807652 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:55.807656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:55.807925 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:55.808117 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:55.808122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.808841 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.808843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:55.809684 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:55.809686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.810086 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.810087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:55.810212 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.810731 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:55.810739 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:55.812227 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:55.813639 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:55.813646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:55.813649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:55.813882 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:55.814104 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:55.814105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:55.814105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:55.814213 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:55.814487 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:55.814494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:55.814639 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 06:35:55.814743 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 06:35:55.814906 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260810 06:35:55.815711 2916 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43619
I20260810 06:35:55.815819 2916 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:55.815964 2916 heartbeater.cc:511] Master 127.30.71.62:43619 requested a full tablet report, sending...
I20260810 06:35:55.816334 2423 ts_manager.cc:194] Registered new tserver with Master: 18c6fdc6a2a14772ba5f3fe70d3c67f4 (127.30.71.3:38137)
I20260810 06:35:55.816668 2423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:51273
I20260810 06:35:55.817761 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:35:55.842640 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.834426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.30.71.1:40163)
0810 06:35:55.834624 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:55.834630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:55.834643 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:35:55.834673 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:55.834675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:55.834692 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:55.834794 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:55.835461 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.836299 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.837167 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.837564 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.837652 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.839962 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:55.839971 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:55.839971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:55.839979 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:55.841411 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.841792 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.841796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.841796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.841858 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.842060 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.842061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.842062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.842205 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:55.842364 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:55.842434 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 06:35:55.842517 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:55.851092 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:55.844238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.30.71.62:43619)
0810 06:35:55.844335 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:55.844337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:55.844347 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:55.844386 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:55.844387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:55.844397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:55.844432 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:55.844847 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.845354 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.845994 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:55.846173 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:55.846226 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:55.848491 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:55.848492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:55.848492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:55.848498 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:55.849682 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.850006 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.850007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.850007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.850075 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:55.850327 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:55.850328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:55.850328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:55.850607 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:55.850711 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:55.850842 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 06:35:55.850948 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 06:35:55.851473 2423 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59970
I20260810 06:35:55.851971 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 2489
I20260810 06:35:55.858134 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 2634
I20260810 06:35:55.870393 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 2776
I20260810 06:35:55.877745 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 2387
2026-08-10T06:35:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2008 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2929](info): set up 2 sockets
Aug 10 06:35:55 dist-test-slave-w206 krb5kdc[2929](info): commencing operation
krb5kdc: starting...
W20260810 06:35:56.963886 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:35:57.092144 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33815
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:33815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:57.175534 2943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:57.175755 2943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:57.175782 2943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:57.177006 2943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:57.177054 2943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:57.177069 2943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:57.177083 2943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:57.177122 2943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:57.178704 2943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:33815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:57.178980 2943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:57.179271 2943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:57.184382 2950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:57.184309 2949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:57.184438 2952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:57.184610 2943 server_base.cc:1034] running on GCE node
I20260810 06:35:57.184919 2943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:57.185233 2943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:57.186482 2943 hybrid_clock.cc:648] HybridClock initialized: now 1786343757186461 us; error 31 us; skew 500 ppm
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:57.189354 2943 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:35:57.189733 2943 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:57.189815 2943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:57.189971 2943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:57.191160 2943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "517ddfb24718443d8da833fde9786f6f"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "bab3ec046164bee0c4bceb0f706c003e"
server_key_iv: "f34c9de7ad5ebd6cba40cfaadd1be352"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.191527 2943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "517ddfb24718443d8da833fde9786f6f"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "bab3ec046164bee0c4bceb0f706c003e"
server_key_iv: "f34c9de7ad5ebd6cba40cfaadd1be352"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.192902 2943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:35:57.193691 2958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.193877 2943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.193979 2943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "517ddfb24718443d8da833fde9786f6f"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "bab3ec046164bee0c4bceb0f706c003e"
server_key_iv: "f34c9de7ad5ebd6cba40cfaadd1be352"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.194068 2943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:57.213030 2943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:57.215822 2943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:57.216038 2943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:57.220713 2943 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:33815
I20260810 06:35:57.220763 3015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:33815 every 8 connection(s)
I20260810 06:35:57.221108 2943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:57.222225 3016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:57.224661 3016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f: Bootstrap starting.
I20260810 06:35:57.225284 3016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:57.225591 3016 log.cc:824] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:57.226353 3016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f: No bootstrap required, opened a new log
I20260810 06:35:57.228224 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 2943
I20260810 06:35:57.228111 3016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517ddfb24718443d8da833fde9786f6f" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33815 } }
I20260810 06:35:57.228308 3016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:57.228332 3016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517ddfb24718443d8da833fde9786f6f, State: Initialized, Role: FOLLOWER
I20260810 06:35:57.228356 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:57.228513 3016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517ddfb24718443d8da833fde9786f6f" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33815 } }
I20260810 06:35:57.228576 3016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:57.228615 31004 external_mini_cluster.cc:1428] Setting key 9099c62e4b4e94caee96c1255a462a14
I20260810 06:35:57.228626 3016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:57.228672 3016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:57.229837 3016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517ddfb24718443d8da833fde9786f6f" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33815 } }
I20260810 06:35:57.229947 3016 leader_election.cc:302] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 517ddfb24718443d8da833fde9786f6f; no voters:
I20260810 06:35:57.230110 3016 leader_election.cc:288] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:57.230170 3021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:57.230325 3021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [term 1 LEADER]: Becoming Leader. State: Replica: 517ddfb24718443d8da833fde9786f6f, State: Running, Role: LEADER
I20260810 06:35:57.230399 3016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:57.230430 3021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517ddfb24718443d8da833fde9786f6f" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33815 } }
I20260810 06:35:57.230785 3024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 517ddfb24718443d8da833fde9786f6f. Latest consensus state: current_term: 1 leader_uuid: "517ddfb24718443d8da833fde9786f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517ddfb24718443d8da833fde9786f6f" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33815 } } }
I20260810 06:35:57.230866 3024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:57.233347 3022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "517ddfb24718443d8da833fde9786f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "517ddfb24718443d8da833fde9786f6f" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33815 } } }
I20260810 06:35:57.233453 3022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f [sys.catalog]: This master's current role is: LEADER
W20260810 06:35:57.233633 3038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:35:57.233673 3038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:35:57.234598 3028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:57.234807 3028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:57.236922 3028 catalog_manager.cc:1347] Generated new cluster ID: 7fe42710548240648beb85ace73b263e
I20260810 06:35:57.237011 3028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:57.244206 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.229178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.30.71.62:33815)
0810 06:35:57.229426 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.229439 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.229483 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:35:57.229614 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.229621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.229678 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.229840 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:57.230690 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.233385 (+ 2695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.234513 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.235242 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.235375 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.238598 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:57.238602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:57.238602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:57.238618 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:57.241640 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.242141 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.242145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.242145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.242319 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.242551 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.242553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.242553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.242956 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:57.243155 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.243762 (+ 607us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.243936 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":119,"threads_started":1}
I20260810 06:35:57.257794 3028 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:57.258675 3028 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:57.274180 3028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f: Generated new TSK 0
I20260810 06:35:57.274526 3028 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:35:57.312278 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33815
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:57.398207 3044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:57.398473 3044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:57.398499 3044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:57.399765 3044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:57.399811 3044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:57.399859 3044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:57.401391 3044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:57.401659 3044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:57.401897 3044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:57.407138 3050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:57.407135 3049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:57.407135 3052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:57.407472 3044 server_base.cc:1034] running on GCE node
I20260810 06:35:57.407732 3044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:57.408001 3044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:57.409178 3044 hybrid_clock.cc:648] HybridClock initialized: now 1786343757409167 us; error 41 us; skew 500 ppm
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:57.411875 3044 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:35:57.412323 3044 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:57.412400 3044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:57.412552 3044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:57.413659 3044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "487ccd0a34d548ebbf7d7b5e44c6e777"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "c0825cf86427b99b6ad519b90be68d6c"
server_key_iv: "c1f7e53c12a2b782201916f1cd57c7bb"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.414032 3044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "487ccd0a34d548ebbf7d7b5e44c6e777"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "c0825cf86427b99b6ad519b90be68d6c"
server_key_iv: "c1f7e53c12a2b782201916f1cd57c7bb"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.415428 3044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:35:57.416141 3060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.416360 3044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.416464 3044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "487ccd0a34d548ebbf7d7b5e44c6e777"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "c0825cf86427b99b6ad519b90be68d6c"
server_key_iv: "c1f7e53c12a2b782201916f1cd57c7bb"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.416538 3044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:57.435999 3044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:57.437994 3044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:57.438161 3044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:57.438416 3044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:57.438731 3044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:57.438786 3044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.438828 3044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:57.438858 3044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.444985 3044 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:43463
I20260810 06:35:57.445035 3178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:43463 every 8 connection(s)
I20260810 06:35:57.445363 3044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:57.448040 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 3044
I20260810 06:35:57.448153 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:57.448362 31004 external_mini_cluster.cc:1428] Setting key eaa876d24e0d93b140ff339321cca746
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:57.455833 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.446149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:36813 (local address 127.30.71.62:33815)
0810 06:35:57.446239 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.446241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.446419 (+ 178us) server_negotiation.cc:407] Connection header received
0810 06:35:57.446880 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.446881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.446894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.446944 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:57.447997 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.448725 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.450142 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.450351 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.450406 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.452823 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:57.452825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:57.452825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:57.452833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:57.454339 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.454773 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.454774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.454775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.454844 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.455142 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.455143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.455144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.455264 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:57.455343 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.455631 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.455727 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:35:57.456002 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.446096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33815 (local address 127.30.71.1:36813)
0810 06:35:57.446287 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:57.446324 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:57.446792 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:57.447022 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:57.447026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:57.447313 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:57.447894 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.447902 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.448798 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.448799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.450054 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.450058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.450437 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.450439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.450585 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.451271 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:57.451280 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:57.452710 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:57.454447 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:57.454455 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:57.454457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:57.454693 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:57.454914 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:57.454915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:57.454915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:57.455063 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:57.455332 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:57.455340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:57.455491 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 06:35:57.455595 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 06:35:57.455757 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260810 06:35:57.456542 3179 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33815
I20260810 06:35:57.456642 3179 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:57.456849 3179 heartbeater.cc:511] Master 127.30.71.62:33815 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:35:57.457391 2978 ts_manager.cc:194] Registered new tserver with Master: 487ccd0a34d548ebbf7d7b5e44c6e777 (127.30.71.1:43463)
I20260810 06:35:57.458004 2978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:36813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:35:57.478276 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33815
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:57.562768 3184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:57.562966 3184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:57.563020 3184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:57.564208 3184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:57.564251 3184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:57.564287 3184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:57.565846 3184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:57.566103 3184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:57.566349 3184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:57.571666 3192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:57.571709 3189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:57.571883 3184 server_base.cc:1034] running on GCE node
W20260810 06:35:57.571754 3190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:57.572213 3184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:57.572515 3184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:57.573716 3184 hybrid_clock.cc:648] HybridClock initialized: now 1786343757573700 us; error 58 us; skew 500 ppm
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:57.576507 3184 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:35:57.576967 3184 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:57.577049 3184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:57.577219 3184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:57.578401 3184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "4ada8185b0644ee89de4c7edfe77921d"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "ee50430d28ec8791f95838aa8f1fe795"
server_key_iv: "4dd760a458d2eef9a1f0dfa351a5f1d2"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.578814 3184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "4ada8185b0644ee89de4c7edfe77921d"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "ee50430d28ec8791f95838aa8f1fe795"
server_key_iv: "4dd760a458d2eef9a1f0dfa351a5f1d2"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.580332 3184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:57.581187 3198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.581363 3184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:57.581471 3184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "4ada8185b0644ee89de4c7edfe77921d"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "ee50430d28ec8791f95838aa8f1fe795"
server_key_iv: "4dd760a458d2eef9a1f0dfa351a5f1d2"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.581547 3184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:57.588716 3184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:57.590466 3184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:57.590626 3184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:57.590853 3184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:57.591218 3184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:57.591279 3184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.591312 3184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:57.591342 3184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.597648 3184 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:45953
I20260810 06:35:57.597750 3319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:45953 every 8 connection(s)
I20260810 06:35:57.597996 3184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:57.604748 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 3184
I20260810 06:35:57.604895 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:57.605134 31004 external_mini_cluster.cc:1428] Setting key c47a692702c6adbbd3721280a535cdbf
I20260810 06:35:57.607892 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.598768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:41235 (local address 127.30.71.62:33815)
0810 06:35:57.598858 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.598861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.599032 (+ 171us) server_negotiation.cc:407] Connection header received
0810 06:35:57.599504 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.599505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.599516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.599598 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:57.600601 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.601122 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.602152 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.602357 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.602445 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.604628 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:57.604630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:57.604630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:57.604638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:57.606278 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.606704 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.606706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.606706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.606772 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.607079 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.607080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.607080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.607241 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:57.607317 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.607662 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.607761 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:35:57.607903 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.598700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33815 (local address 127.30.71.2:41235)
0810 06:35:57.598869 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:57.598906 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:57.599410 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:57.599682 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:57.599687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:57.599991 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:57.600494 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.600498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.601229 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.601230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.602039 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.602041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.602454 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.602455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.602584 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.603096 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:57.603104 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:57.604525 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:57.606368 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:57.606375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:57.606377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:57.606608 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:57.606848 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:57.606849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:57.606849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:57.607020 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:57.607326 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:57.607334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:57.607513 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 06:35:57.607600 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:35:57.607730 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":97,"threads_started":1}
I20260810 06:35:57.608585 3320 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33815
I20260810 06:35:57.608686 3320 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:57.608860 3320 heartbeater.cc:511] Master 127.30.71.62:33815 requested a full tablet report, sending...
I20260810 06:35:57.609228 2978 ts_manager.cc:194] Registered new tserver with Master: 4ada8185b0644ee89de4c7edfe77921d (127.30.71.2:45953)
I20260810 06:35:57.609624 2978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:41235
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:35:57.633853 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33815
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:35:57.717250 3325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:57.717463 3325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:57.717486 3325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:57.718581 3325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:57.718614 3325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:57.718641 3325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:57.720149 3325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:57.720342 3325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:57.720513 3325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:57.725821 3330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:57.725850 3331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:57.725878 3325 server_base.cc:1034] running on GCE node
W20260810 06:35:57.725826 3333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:57.726241 3325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:57.726527 3325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:57.727674 3325 hybrid_clock.cc:648] HybridClock initialized: now 1786343757727660 us; error 27 us; skew 500 ppm
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:35:57.730369 3325 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:35:57.730762 3325 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:57.730865 3325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:57.731042 3325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:57.732252 3325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "be14769e174b4ad3940fbd3f50ed6e7d"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "7a22b0ee786873291951ed00a4bf73c4"
server_key_iv: "984d6c93f80c69259f93133e68236d4b"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.732640 3325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "be14769e174b4ad3940fbd3f50ed6e7d"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "7a22b0ee786873291951ed00a4bf73c4"
server_key_iv: "984d6c93f80c69259f93133e68236d4b"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.734088 3325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:57.734864 3339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.735088 3325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:57.735200 3325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "be14769e174b4ad3940fbd3f50ed6e7d"
format_stamp: "Formatted at 2026-08-10 06:35:57 on dist-test-slave-w206"
server_key: "7a22b0ee786873291951ed00a4bf73c4"
server_key_iv: "984d6c93f80c69259f93133e68236d4b"
server_key_version: "encryptionkey@0"
I20260810 06:35:57.735278 3325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:57.753597 3325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:57.755743 3325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:57.755919 3325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:57.756181 3325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:57.756531 3325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:57.756587 3325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.756623 3325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:57.756670 3325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:57.762633 3325 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:37663
I20260810 06:35:57.762712 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:37663 every 8 connection(s)
I20260810 06:35:57.763055 3325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:35:57.769271 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 3325
I20260810 06:35:57.769359 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
Aug 10 06:35:57 dist-test-slave-w206 krb5kdc[2929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343757, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:35:57.769586 31004 external_mini_cluster.cc:1428] Setting key 50089ac452425903337bc72a8e9559ee
I20260810 06:35:57.773312 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.763887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:36923 (local address 127.30.71.62:33815)
0810 06:35:57.763961 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.763965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.764174 (+ 209us) server_negotiation.cc:407] Connection header received
0810 06:35:57.764685 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.764686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.764696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.764748 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:57.765674 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.766171 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.767246 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.767452 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.767512 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.770007 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:57.770009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:57.770009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:57.770018 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:57.771719 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.772225 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.772227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.772227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.772298 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.772574 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.772575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.772576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.772711 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:57.772792 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.773104 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.773208 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:35:57.773545 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.763866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33815 (local address 127.30.71.3:36923)
0810 06:35:57.764048 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:57.764083 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:57.764599 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:57.764801 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:57.764804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:57.765073 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:57.765589 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.765594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.766310 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.766311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.767163 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.767166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.767583 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.767584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.767705 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.768265 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:35:57.768273 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:57.769895 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:57.771845 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:57.771855 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:57.771858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:57.772136 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:57.772369 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:35:57.772370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:35:57.772370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:35:57.772502 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:35:57.772781 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:57.772790 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:57.772956 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 06:35:57.773079 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:35:57.773282 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":115,"threads_started":1}
I20260810 06:35:57.774056 3460 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33815
I20260810 06:35:57.774150 3460 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:57.774430 3460 heartbeater.cc:511] Master 127.30.71.62:33815 requested a full tablet report, sending...
I20260810 06:35:57.774900 2978 ts_manager.cc:194] Registered new tserver with Master: be14769e174b4ad3940fbd3f50ed6e7d (127.30.71.3:37663)
I20260810 06:35:57.775323 2978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:36923
I20260810 06:35:57.781575 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:35:57.789268 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.783045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.30.71.62:33815)
0810 06:35:57.783128 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.783131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.783229 (+ 98us) server_negotiation.cc:407] Connection header received
0810 06:35:57.783270 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.783270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.783279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.783329 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:57.783877 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.784492 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.785248 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.785476 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.785537 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.786715 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:57.786717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:57.786717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:35:57.786724 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:57.788253 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.788550 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.788551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.788551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.788603 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:35:57.788763 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:35:57.788763 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:35:57.788764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:35:57.788898 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:57.788971 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.789063 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.789122 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:35:57.791087 2978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:57.792676 2978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:57.799618 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.795394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:43463 (local address 127.0.0.1:33596)
0810 06:35:57.795626 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:57.795637 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:57.795698 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:57.796318 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:57.796319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:57.796321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:57.796358 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.796359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.797639 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.797640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.798557 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.798560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.799323 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.799324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.799402 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.799416 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:35:57.799489 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:35:57.799541 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260810 06:35:57.799628 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.795189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.30.71.1:43463)
0810 06:35:57.795434 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.795440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.795693 (+ 253us) server_negotiation.cc:407] Connection header received
0810 06:35:57.795799 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.795801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.795817 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.796232 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:57.796451 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.797536 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.798648 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.799236 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.799323 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.799386 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.799511 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.799548 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
I20260810 06:35:57.799927 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.795071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:45953 (local address 127.0.0.1:55104)
0810 06:35:57.795393 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:57.795440 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:57.795515 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:57.795924 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:57.795925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:57.795926 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:57.796099 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.796102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.797114 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.797115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.798580 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.798582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.799594 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.799594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.799670 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.799714 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 06:35:57.799785 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:35:57.799835 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":38,"thread_start_us":81,"threads_started":1}
I20260810 06:35:57.800103 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.795074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:37663 (local address 127.0.0.1:53080)
0810 06:35:57.795526 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:57.795534 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:57.795574 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:57.795791 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:57.795794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:57.795796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:57.795871 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.795873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.797237 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.797237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.798579 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:57.798582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.799590 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.799592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:57.799663 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.799668 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:35:57.799866 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 06:35:57.799914 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":72,"thread_start_us":71,"threads_started":1}
I20260810 06:35:57.800189 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.795189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.30.71.2:45953)
0810 06:35:57.795530 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.795534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.795546 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:35:57.795829 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.795831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.795847 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.795923 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:57.796195 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.797012 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.798706 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.799428 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.799497 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.799551 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.800055 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.800099 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":99,"threads_started":1}
I20260810 06:35:57.800311 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:57.795359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.30.71.3:37663)
0810 06:35:57.795564 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:57.795568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:57.795577 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:35:57.795669 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:57.795671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:57.795690 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:57.795788 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:57.795967 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.797129 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.798689 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:57.799425 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:57.799497 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:57.799551 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:57.799976 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 06:35:57.800218 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":102,"threads_started":1}
I20260810 06:35:57.800422 3110 tablet_service.cc:1467] Processing CreateTablet for tablet ae2e6a77513f46ed9fb580e07051a7d8 (DEFAULT_TABLE table=test-table [id=f3c97a6d5426452abdb1d5008524e258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:57.800817 3110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae2e6a77513f46ed9fb580e07051a7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:57.800966 3390 tablet_service.cc:1467] Processing CreateTablet for tablet ae2e6a77513f46ed9fb580e07051a7d8 (DEFAULT_TABLE table=test-table [id=f3c97a6d5426452abdb1d5008524e258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:57.801249 3390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae2e6a77513f46ed9fb580e07051a7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:57.801249 3250 tablet_service.cc:1467] Processing CreateTablet for tablet ae2e6a77513f46ed9fb580e07051a7d8 (DEFAULT_TABLE table=test-table [id=f3c97a6d5426452abdb1d5008524e258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:57.801555 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae2e6a77513f46ed9fb580e07051a7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:57.804050 3477 tablet_bootstrap.cc:492] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Bootstrap starting.
I20260810 06:35:57.804556 3478 tablet_bootstrap.cc:492] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: Bootstrap starting.
I20260810 06:35:57.804653 3477 tablet_bootstrap.cc:654] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:57.805088 3479 tablet_bootstrap.cc:492] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Bootstrap starting.
I20260810 06:35:57.805243 3478 tablet_bootstrap.cc:654] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:57.805286 3477 log.cc:824] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:57.805800 3479 tablet_bootstrap.cc:654] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:57.805884 3478 log.cc:824] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:57.806222 3477 tablet_bootstrap.cc:492] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: No bootstrap required, opened a new log
I20260810 06:35:57.806326 3477 ts_tablet_manager.cc:1397] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:35:57.806506 3479 log.cc:824] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:57.806643 3478 tablet_bootstrap.cc:492] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: No bootstrap required, opened a new log
I20260810 06:35:57.806718 3478 ts_tablet_manager.cc:1397] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:35:57.807379 3479 tablet_bootstrap.cc:492] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: No bootstrap required, opened a new log
I20260810 06:35:57.807451 3479 ts_tablet_manager.cc:1397] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:35:57.808418 3478 raft_consensus.cc:348] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:57.808480 3477 raft_consensus.cc:348] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:57.808588 3478 raft_consensus.cc:374] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:57.808636 3478 raft_consensus.cc:729] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487ccd0a34d548ebbf7d7b5e44c6e777, State: Initialized, Role: FOLLOWER
I20260810 06:35:57.808640 3477 raft_consensus.cc:374] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:57.808678 3477 raft_consensus.cc:729] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be14769e174b4ad3940fbd3f50ed6e7d, State: Initialized, Role: FOLLOWER
I20260810 06:35:57.808769 3478 consensus_queue.cc:260] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:57.808818 3477 consensus_queue.cc:260] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:57.809099 3478 ts_tablet_manager.cc:1428] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:35:57.809115 3477 ts_tablet_manager.cc:1428] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:35:57.809139 3460 heartbeater.cc:503] Master 127.30.71.62:33815 was elected leader, sending a full tablet report...
I20260810 06:35:57.809214 3179 heartbeater.cc:503] Master 127.30.71.62:33815 was elected leader, sending a full tablet report...
I20260810 06:35:57.809675 3479 raft_consensus.cc:348] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:57.809862 3479 raft_consensus.cc:374] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:57.809900 3479 raft_consensus.cc:729] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ada8185b0644ee89de4c7edfe77921d, State: Initialized, Role: FOLLOWER
I20260810 06:35:57.810017 3479 consensus_queue.cc:260] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:57.810359 3320 heartbeater.cc:503] Master 127.30.71.62:33815 was elected leader, sending a full tablet report...
I20260810 06:35:57.810359 3479 ts_tablet_manager.cc:1428] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260810 06:35:57.848569 3321 tablet.cc:2369] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:35:57.946146 3180 tablet.cc:2369] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:35:58.013677 3461 tablet.cc:2369] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:35:58.101925 3484 raft_consensus.cc:482] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:58.102113 3484 raft_consensus.cc:504] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:58.102555 3484 leader_election.cc:288] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ada8185b0644ee89de4c7edfe77921d (127.30.71.2:45953), 487ccd0a34d548ebbf7d7b5e44c6e777 (127.30.71.1:43463)
I20260810 06:35:58.105484 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.102769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:43463 (local address 127.30.71.3:42963)
0810 06:35:58.102837 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.102850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.102897 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.103141 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.103143 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.103145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:58.103221 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:58.103228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.103884 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.103885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:58.104648 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:58.104650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.105197 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.105199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:58.105282 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:58.105286 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.105368 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.105411 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:35:58.105484 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.102853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:42963 (local address 127.30.71.1:43463)
0810 06:35:58.102933 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.102936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.102946 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:58.103027 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.103028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.103041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.103133 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:58.103310 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.103794 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.104752 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.105113 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.105198 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:58.105209 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.105376 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.105415 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:35:58.105607 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.102870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:51539 (local address 127.30.71.2:45953)
0810 06:35:58.102933 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.102935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.103170 (+ 235us) server_negotiation.cc:407] Connection header received
0810 06:35:58.103230 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.103231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.103245 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.103335 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:35:58.103492 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.103955 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.104906 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.105263 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.105347 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:58.105356 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.105511 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.105544 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:35:58.105619 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.102769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:45953 (local address 127.30.71.3:51539)
0810 06:35:58.103085 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.103099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.103174 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.103341 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.103342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.103343 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:35:58.103417 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:58.103418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.104050 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.104051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:58.104814 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:58.104816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.105350 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.105351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:58.105425 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:58.105429 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.105493 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.105534 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":115,"thread_start_us":141,"threads_started":1}
I20260810 06:35:58.105938 3132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2e6a77513f46ed9fb580e07051a7d8" candidate_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" is_pre_election: true
I20260810 06:35:58.106003 3270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2e6a77513f46ed9fb580e07051a7d8" candidate_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ada8185b0644ee89de4c7edfe77921d" is_pre_election: true
I20260810 06:35:58.106112 3132 raft_consensus.cc:2454] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be14769e174b4ad3940fbd3f50ed6e7d in term 0.
I20260810 06:35:58.106164 3270 raft_consensus.cc:2454] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be14769e174b4ad3940fbd3f50ed6e7d in term 0.
I20260810 06:35:58.106387 3345 leader_election.cc:302] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487ccd0a34d548ebbf7d7b5e44c6e777, be14769e174b4ad3940fbd3f50ed6e7d; no voters:
I20260810 06:35:58.106588 3484 raft_consensus.cc:2781] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:58.106681 3484 raft_consensus.cc:482] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:58.106714 3484 raft_consensus.cc:3037] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:58.107514 3484 raft_consensus.cc:504] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:58.107677 3484 leader_election.cc:288] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [CANDIDATE]: Term 1 election: Requested vote from peers 4ada8185b0644ee89de4c7edfe77921d (127.30.71.2:45953), 487ccd0a34d548ebbf7d7b5e44c6e777 (127.30.71.1:43463)
I20260810 06:35:58.107888 3132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2e6a77513f46ed9fb580e07051a7d8" candidate_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777"
I20260810 06:35:58.107892 3270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2e6a77513f46ed9fb580e07051a7d8" candidate_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ada8185b0644ee89de4c7edfe77921d"
I20260810 06:35:58.107969 3132 raft_consensus.cc:3037] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:58.107992 3270 raft_consensus.cc:3037] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:58.108713 3270 raft_consensus.cc:2454] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be14769e174b4ad3940fbd3f50ed6e7d in term 1.
I20260810 06:35:58.108742 3132 raft_consensus.cc:2454] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be14769e174b4ad3940fbd3f50ed6e7d in term 1.
I20260810 06:35:58.108911 3342 leader_election.cc:302] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ada8185b0644ee89de4c7edfe77921d, be14769e174b4ad3940fbd3f50ed6e7d; no voters:
I20260810 06:35:58.109071 3484 raft_consensus.cc:2781] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:58.109277 3484 raft_consensus.cc:686] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 1 LEADER]: Becoming Leader. State: Replica: be14769e174b4ad3940fbd3f50ed6e7d, State: Running, Role: LEADER
I20260810 06:35:58.109364 3484 consensus_queue.cc:237] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } }
I20260810 06:35:58.110265 2977 catalog_manager.cc:5495] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d reported cstate change: term changed from 0 to 1, leader changed from <none> to be14769e174b4ad3940fbd3f50ed6e7d (127.30.71.3). New cstate: current_term: 1 leader_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be14769e174b4ad3940fbd3f50ed6e7d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37663 } health_report { overall_health: HEALTHY } } }
I20260810 06:35:58.153664 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.151056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53094 (local address 127.30.71.3:37663)
0810 06:35:58.151114 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.151116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.151126 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:58.151204 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.151205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.151230 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.151356 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:35:58.151524 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.152005 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.152825 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:58.153033 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:58.153111 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:58.153301 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:35:58.153414 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:35:58.153512 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.153531 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.153553 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:35:58.155702 3270 raft_consensus.cc:1261] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 1 FOLLOWER]: Refusing update from remote peer be14769e174b4ad3940fbd3f50ed6e7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:58.155704 3132 raft_consensus.cc:1261] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 1 FOLLOWER]: Refusing update from remote peer be14769e174b4ad3940fbd3f50ed6e7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:58.156073 3484 consensus_queue.cc:1035] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ada8185b0644ee89de4c7edfe77921d" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:58.156220 3487 consensus_queue.cc:1035] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "487ccd0a34d548ebbf7d7b5e44c6e777" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:58.159554 3493 mvcc.cc:205] Tried to move back new op lower bound from 7316864033402798080 to 7316864033216663552. Current Snapshot: MvccSnapshot[applied={T|T < 7316864033402798080}]
I20260810 06:35:58.160956 3492 mvcc.cc:205] Tried to move back new op lower bound from 7316864033402798080 to 7316864033216663552. Current Snapshot: MvccSnapshot[applied={T|T < 7316864033402798080 or (T in {7316864033402798080})}]
I20260810 06:35:58.161237 2978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:58.161345 2978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:58.162214 2978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 517ddfb24718443d8da833fde9786f6f: Sending DeleteTablet for 3 replicas of tablet ae2e6a77513f46ed9fb580e07051a7d8
I20260810 06:35:58.162609 3250 tablet_service.cc:1514] Processing DeleteTablet for tablet ae2e6a77513f46ed9fb580e07051a7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:58 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:55104
I20260810 06:35:58.162591 3110 tablet_service.cc:1514] Processing DeleteTablet for tablet ae2e6a77513f46ed9fb580e07051a7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:58 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:33596
I20260810 06:35:58.162626 3390 tablet_service.cc:1514] Processing DeleteTablet for tablet ae2e6a77513f46ed9fb580e07051a7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:58 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:53080
I20260810 06:35:58.162887 3504 tablet_replica.cc:330] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: stopping tablet replica
I20260810 06:35:58.162946 3502 tablet_replica.cc:330] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777: stopping tablet replica
I20260810 06:35:58.163079 3502 raft_consensus.cc:2229] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:58.163084 3504 raft_consensus.cc:2229] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:58.163201 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 3044
I20260810 06:35:58.163213 3502 raft_consensus.cc:2258] T ae2e6a77513f46ed9fb580e07051a7d8 P 487ccd0a34d548ebbf7d7b5e44c6e777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:58.164861 3501 tablet_replica.cc:330] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: stopping tablet replica
I20260810 06:35:58.164975 3501 raft_consensus.cc:2229] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:58.165098 3501 raft_consensus.cc:2258] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:58.165356 3504 raft_consensus.cc:2258] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:58.166142 3501 ts_tablet_manager.cc:1905] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:58.167272 3504 ts_tablet_manager.cc:1905] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:58.168287 3501 ts_tablet_manager.cc:1918] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:35:58.168377 3501 log.cc:1196] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/ae2e6a77513f46ed9fb580e07051a7d8
I20260810 06:35:58.168655 3501 ts_tablet_manager.cc:1939] T ae2e6a77513f46ed9fb580e07051a7d8 P 4ada8185b0644ee89de4c7edfe77921d: Deleting consensus metadata
I20260810 06:35:58.169193 2961 catalog_manager.cc:4841] TS 4ada8185b0644ee89de4c7edfe77921d (127.30.71.2:45953): tablet ae2e6a77513f46ed9fb580e07051a7d8 (table test-table [id=f3c97a6d5426452abdb1d5008524e258]) successfully deleted
W20260810 06:35:58.170805 2964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:43463 (error 108)
I20260810 06:35:58.171195 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 3184
I20260810 06:35:58.172873 3504 ts_tablet_manager.cc:1918] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:35:58.172974 3504 log.cc:1196] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/ae2e6a77513f46ed9fb580e07051a7d8
I20260810 06:35:58.173316 3504 ts_tablet_manager.cc:1939] T ae2e6a77513f46ed9fb580e07051a7d8 P be14769e174b4ad3940fbd3f50ed6e7d: Deleting consensus metadata
I20260810 06:35:58.174037 2960 catalog_manager.cc:4841] TS be14769e174b4ad3940fbd3f50ed6e7d (127.30.71.3:37663): tablet ae2e6a77513f46ed9fb580e07051a7d8 (table test-table [id=f3c97a6d5426452abdb1d5008524e258]) successfully deleted
I20260810 06:35:58.175356 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.175181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:43463 (local address 127.0.0.1:33606)
0810 06:35:58.175250 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.175318 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:35:58.175498 2964 catalog_manager.cc:4584] TS 487ccd0a34d548ebbf7d7b5e44c6e777 (127.30.71.1:43463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae2e6a77513f46ed9fb580e07051a7d8: Network error: Client connection negotiation failed: client connection to 127.30.71.1:43463: connect: Connection refused (error 111)
I20260810 06:35:58.178599 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 3325
I20260810 06:35:58.185079 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 2943
2026-08-10T06:35:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2308 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T06:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:58Z Disabled control of system clock
I20260810 06:35:58.216583 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:39753
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:32929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:39753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:35:58.306957 3509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:58.307211 3509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:58.307233 3509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:58.308331 3509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:58.308364 3509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:58.308377 3509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:58.308390 3509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:58.308403 3509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:35:58.309671 3509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:32929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:39753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:39753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:58.309877 3509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:58.310106 3509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:58.315127 3516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.315127 3518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.315138 3515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:58.315248 3509 server_base.cc:1034] running on GCE node
I20260810 06:35:58.315636 3509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:58.315953 3509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:58.317124 3509 hybrid_clock.cc:648] HybridClock initialized: now 1786343758317120 us; error 40 us; skew 500 ppm
I20260810 06:35:58.318460 3509 webserver.cc:460] Webserver started at http://127.30.71.62:44509/ using document root <none> and password file <none>
I20260810 06:35:58.318710 3509 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:58.318800 3509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:58.318964 3509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:58.320182 3509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "4d4a2666e13e43cb9d40df9ac3550735"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "2f6a69da8b4c679451c35483e58c8c13"
server_key_iv: "914ac9e5bd431cc52b146a16c01acc4d"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.320570 3509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "4d4a2666e13e43cb9d40df9ac3550735"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "2f6a69da8b4c679451c35483e58c8c13"
server_key_iv: "914ac9e5bd431cc52b146a16c01acc4d"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.322050 3509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:35:58.322857 3525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.323117 3509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:35:58.323233 3509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "4d4a2666e13e43cb9d40df9ac3550735"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "2f6a69da8b4c679451c35483e58c8c13"
server_key_iv: "914ac9e5bd431cc52b146a16c01acc4d"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.323318 3509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:58.326654 3509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:58.326934 3509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:58.327109 3509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:58.331190 3509 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:39753
I20260810 06:35:58.331226 3578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:39753 every 8 connection(s)
I20260810 06:35:58.331646 3509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:58.332435 3579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:58.335525 3579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735: Bootstrap starting.
I20260810 06:35:58.336067 3579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:58.336318 3579 log.cc:824] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:58.336964 3579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735: No bootstrap required, opened a new log
I20260810 06:35:58.338512 3579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4a2666e13e43cb9d40df9ac3550735" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39753 } }
I20260810 06:35:58.338651 3579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:58.338673 3579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4a2666e13e43cb9d40df9ac3550735, State: Initialized, Role: FOLLOWER
I20260810 06:35:58.338821 3579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4a2666e13e43cb9d40df9ac3550735" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39753 } }
I20260810 06:35:58.338905 3579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:58.338929 3579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:58.338958 3579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:58.339547 3579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4a2666e13e43cb9d40df9ac3550735" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39753 } }
I20260810 06:35:58.339671 3579 leader_election.cc:302] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d4a2666e13e43cb9d40df9ac3550735; no voters:
I20260810 06:35:58.339864 3579 leader_election.cc:288] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:58.339926 3582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:58.340142 3582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [term 1 LEADER]: Becoming Leader. State: Replica: 4d4a2666e13e43cb9d40df9ac3550735, State: Running, Role: LEADER
I20260810 06:35:58.340165 3579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:58.340234 3582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4a2666e13e43cb9d40df9ac3550735" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39753 } }
I20260810 06:35:58.340664 3585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d4a2666e13e43cb9d40df9ac3550735. Latest consensus state: current_term: 1 leader_uuid: "4d4a2666e13e43cb9d40df9ac3550735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4a2666e13e43cb9d40df9ac3550735" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39753 } } }
I20260810 06:35:58.340777 3585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:58.341068 3594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:58.341089 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d4a2666e13e43cb9d40df9ac3550735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d4a2666e13e43cb9d40df9ac3550735" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39753 } } }
I20260810 06:35:58.341207 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:58.341691 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 3509
I20260810 06:35:58.341814 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:58.342020 31004 external_mini_cluster.cc:1428] Setting key 054043f0a1664dbe7be97ea9cfa6a639
I20260810 06:35:58.342338 3594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:58.344074 3594 catalog_manager.cc:1347] Generated new cluster ID: 5b3f4c91dea7454bb56a84fa188f83be
I20260810 06:35:58.344132 3594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:58.344907 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.342511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.30.71.62:39753)
0810 06:35:58.342769 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.342778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.342956 (+ 178us) server_negotiation.cc:407] Connection header received
0810 06:35:58.343161 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.343168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.343215 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.343359 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.344441 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.344444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.344445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.344449 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.344503 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.344514 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.344587 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.344660 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.344720 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260810 06:35:58.369266 3594 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:58.369515 3594 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:58.376631 3594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735: Generated new TSK 0
I20260810 06:35:58.376883 3594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:35:58.380510 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39753
--builtin_ntp_servers=127.30.71.20:32929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:35:58.463538 3606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:58.463783 3606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:58.463819 3606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:58.464993 3606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:58.465061 3606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:58.465106 3606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:35:58.466485 3606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:32929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:58.466758 3606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:58.467124 3606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:58.472385 3614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.472357 3612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.472401 3611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:58.472915 3606 server_base.cc:1034] running on GCE node
I20260810 06:35:58.473160 3606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:58.473423 3606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:58.474624 3606 hybrid_clock.cc:648] HybridClock initialized: now 1786343758474613 us; error 35 us; skew 500 ppm
I20260810 06:35:58.476083 3606 webserver.cc:460] Webserver started at http://127.30.71.1:38359/ using document root <none> and password file <none>
I20260810 06:35:58.476346 3606 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:58.476421 3606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:58.476573 3606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:58.477753 3606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "ee193bcfe36a4026bedc4d5b51398f3d"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "6e38ea6f5c372eda80d542ee33111759"
server_key_iv: "5cc92afd1a5212719d77bae35545a319"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.478127 3606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "ee193bcfe36a4026bedc4d5b51398f3d"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "6e38ea6f5c372eda80d542ee33111759"
server_key_iv: "5cc92afd1a5212719d77bae35545a319"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.479773 3606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:35:58.480604 3620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.480773 3606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:35:58.480888 3606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "ee193bcfe36a4026bedc4d5b51398f3d"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "6e38ea6f5c372eda80d542ee33111759"
server_key_iv: "5cc92afd1a5212719d77bae35545a319"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.480968 3606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:58.484427 3606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:58.484705 3606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:58.484855 3606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:58.485064 3606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:58.485383 3606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:58.485445 3606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.485483 3606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:58.485512 3606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.491559 3606 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:40717
I20260810 06:35:58.491649 3739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:40717 every 8 connection(s)
I20260810 06:35:58.491997 3606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:35:58.495329 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.492753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:44351 (local address 127.30.71.62:39753)
0810 06:35:58.492830 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.492832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.493049 (+ 217us) server_negotiation.cc:407] Connection header received
0810 06:35:58.493578 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.493580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.493589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.493667 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.495071 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.495072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.495073 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.495077 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.495095 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.495099 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.495165 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.495253 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.495270 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:35:58.495513 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.492760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39753 (local address 127.30.71.1:44351)
0810 06:35:58.492931 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.492966 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.493452 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.493731 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.493734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.493743 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.494830 (+ 1087us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.494830 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.494861 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.494891 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.495172 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.495177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.495180 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.495244 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.495297 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260810 06:35:58.496140 3740 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39753
I20260810 06:35:58.496248 3740 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:58.496246 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 3606
I20260810 06:35:58.496359 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:35:58.496430 3740 heartbeater.cc:511] Master 127.30.71.62:39753 requested a full tablet report, sending...
I20260810 06:35:58.496611 31004 external_mini_cluster.cc:1428] Setting key 4412c045761d04f0aaff68c4193b3d73
I20260810 06:35:58.497064 3542 ts_manager.cc:194] Registered new tserver with Master: ee193bcfe36a4026bedc4d5b51398f3d (127.30.71.1:40717)
I20260810 06:35:58.497999 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39753
--builtin_ntp_servers=127.30.71.20:32929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:35:58.582254 3743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:58.582465 3743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:58.582489 3743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:58.583741 3743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:58.583791 3743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:58.583817 3743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:35:58.585175 3743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:32929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:58.585379 3743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:58.585633 3743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:58.590685 3748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:58.590849 3743 server_base.cc:1034] running on GCE node
W20260810 06:35:58.590679 3749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.590683 3751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:58.591141 3743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:58.591418 3743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:58.592563 3743 hybrid_clock.cc:648] HybridClock initialized: now 1786343758592553 us; error 32 us; skew 500 ppm
I20260810 06:35:58.593837 3743 webserver.cc:460] Webserver started at http://127.30.71.2:36299/ using document root <none> and password file <none>
I20260810 06:35:58.594105 3743 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:58.594152 3743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:58.594362 3743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:58.595589 3743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "b89aac0303bc45a3927d9adb8ddcf0ec"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "6ac819e565068621f1ad0334ae9bebda"
server_key_iv: "74e2e5269cc901dcd68b45c7a5928099"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.595992 3743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "b89aac0303bc45a3927d9adb8ddcf0ec"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "6ac819e565068621f1ad0334ae9bebda"
server_key_iv: "74e2e5269cc901dcd68b45c7a5928099"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.597811 3743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:35:58.598659 3757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.598871 3743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:58.598976 3743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "b89aac0303bc45a3927d9adb8ddcf0ec"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "6ac819e565068621f1ad0334ae9bebda"
server_key_iv: "74e2e5269cc901dcd68b45c7a5928099"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.599104 3743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:58.602548 3743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:58.602842 3743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:58.603018 3743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:58.603243 3743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:58.603560 3743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:58.603627 3743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.603668 3743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:58.603696 3743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.609864 3743 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:34103
I20260810 06:35:58.609915 3875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:34103 every 8 connection(s)
I20260810 06:35:58.610271 3743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:35:58.613338 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 3743
I20260810 06:35:58.613471 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:35:58.613703 31004 external_mini_cluster.cc:1428] Setting key 40e233cf4f2cac0bdb87291e84b1c1f0
I20260810 06:35:58.613721 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.611080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:43209 (local address 127.30.71.62:39753)
0810 06:35:58.611148 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.611150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.611380 (+ 230us) server_negotiation.cc:407] Connection header received
0810 06:35:58.611961 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.611963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.611971 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.612023 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.613375 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.613377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.613377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.613382 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.613403 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.613408 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.613463 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.613623 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.613643 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:35:58.613891 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.611044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39753 (local address 127.30.71.2:43209)
0810 06:35:58.611248 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.611285 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.611827 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.612106 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.612110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.612119 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.613208 (+ 1089us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.613209 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.613243 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.613273 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.613488 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.613494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.613497 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.613578 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.613649 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260810 06:35:58.614336 3876 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39753
I20260810 06:35:58.614457 3876 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:58.614624 3876 heartbeater.cc:511] Master 127.30.71.62:39753 requested a full tablet report, sending...
I20260810 06:35:58.614914 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39753
--builtin_ntp_servers=127.30.71.20:32929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:35:58.616395 3542 ts_manager.cc:194] Registered new tserver with Master: b89aac0303bc45a3927d9adb8ddcf0ec (127.30.71.2:34103)
W20260810 06:35:58.705910 3879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:58.706199 3879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:58.706244 3879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:58.707660 3879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:58.707729 3879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:58.707775 3879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:35:58.709313 3879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:32929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:58.709609 3879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:58.709956 3879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:58.715575 3886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.715588 3885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.715597 3888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:58.715935 3879 server_base.cc:1034] running on GCE node
I20260810 06:35:58.716214 3879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:58.716471 3879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:58.717638 3879 hybrid_clock.cc:648] HybridClock initialized: now 1786343758717621 us; error 39 us; skew 500 ppm
I20260810 06:35:58.719046 3879 webserver.cc:460] Webserver started at http://127.30.71.3:45295/ using document root <none> and password file <none>
I20260810 06:35:58.719339 3879 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:58.719414 3879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:58.719566 3879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:58.720786 3879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "8511e161102f45d0a9a5f8c958717e60"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "b326ab22ed4459e7b0dca1a6e5524718"
server_key_iv: "20eaaf4060ea13c3acd1652ec022c75e"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.721216 3879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "8511e161102f45d0a9a5f8c958717e60"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "b326ab22ed4459e7b0dca1a6e5524718"
server_key_iv: "20eaaf4060ea13c3acd1652ec022c75e"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.722777 3879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:35:58.723661 3895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.723878 3879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:58.723984 3879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "8511e161102f45d0a9a5f8c958717e60"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "b326ab22ed4459e7b0dca1a6e5524718"
server_key_iv: "20eaaf4060ea13c3acd1652ec022c75e"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.724074 3879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:58.727694 3879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:58.727993 3879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:58.728143 3879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:58.728370 3879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:35:58.728698 3879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:35:58.728763 3879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.728807 3879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:35:58.728832 3879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.735049 3879 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:38033
I20260810 06:35:58.735116 4013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:38033 every 8 connection(s)
I20260810 06:35:58.735500 3879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:35:58.738667 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.736382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:33963 (local address 127.30.71.62:39753)
0810 06:35:58.736428 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.736430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.736705 (+ 275us) server_negotiation.cc:407] Connection header received
0810 06:35:58.737146 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.737147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.737153 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.737223 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.738416 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.738418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.738418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.738422 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.738448 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.738453 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.738536 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.738583 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.738602 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:35:58.738878 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.736342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39753 (local address 127.30.71.3:33963)
0810 06:35:58.736537 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.736580 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.737059 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.737280 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.737283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.737291 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.738224 (+ 933us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.738224 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.738256 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.738289 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.738533 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.738536 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.738538 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.738588 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.738638 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260810 06:35:58.739529 4014 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39753
I20260810 06:35:58.739672 4014 heartbeater.cc:464] Registering TS with master...
I20260810 06:35:58.739848 4014 heartbeater.cc:511] Master 127.30.71.62:39753 requested a full tablet report, sending...
I20260810 06:35:58.740347 3542 ts_manager.cc:194] Registered new tserver with Master: 8511e161102f45d0a9a5f8c958717e60 (127.30.71.3:38033)
I20260810 06:35:58.740512 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 3879
I20260810 06:35:58.740669 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:35:58.740911 31004 external_mini_cluster.cc:1428] Setting key 990c8108c76e73cd9af68b8ccf786d32
I20260810 06:35:58.741995 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:35:58.744614 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.743462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.30.71.62:39753)
0810 06:35:58.743548 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.743550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.743618 (+ 68us) server_negotiation.cc:407] Connection header received
0810 06:35:58.743657 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.743658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.743663 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.743709 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.744386 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.744387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.744387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.744390 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.744401 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.744405 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.744442 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.744495 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.744507 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:35:58.746711 3542 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:35:58.748342 3542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:35:58.753365 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.750922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:34103 (local address 127.0.0.1:54972)
0810 06:35:58.751146 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.751197 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.751273 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.752000 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.752007 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.752015 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.752748 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.752749 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.752768 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.752798 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.753105 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.753111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.753115 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.753203 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.753219 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":123,"threads_started":1}
I20260810 06:35:58.753396 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.751210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38033 (local address 127.0.0.1:39606)
0810 06:35:58.751711 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.751723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.751784 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.752004 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.752009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.752016 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.752837 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.752838 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.752847 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.752864 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.753176 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.753177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.753178 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.753244 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.753261 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":99,"threads_started":1}
I20260810 06:35:58.753464 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.751095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.30.71.2:34103)
0810 06:35:58.751467 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.751472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.751486 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:35:58.751519 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.751522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.751534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.752317 (+ 783us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.752907 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.752916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.752917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.752923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.752966 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.752999 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.753075 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.753337 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.753357 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":193,"threads_started":1}
I20260810 06:35:58.753696 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.751336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39606 (local address 127.30.71.3:38033)
0810 06:35:58.751627 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.751632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.751793 (+ 161us) server_negotiation.cc:407] Connection header received
0810 06:35:58.751849 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.751852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.751863 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.751984 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.752974 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.752982 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.752983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.752988 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.753031 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.753060 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.753139 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.753535 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.753559 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260810 06:35:58.754447 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.751419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:40717 (local address 127.0.0.1:51910)
0810 06:35:58.751717 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.751728 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.751774 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.752858 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.752859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.752862 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.753486 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.753487 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.753494 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.753507 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.754283 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.754284 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.754285 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.754358 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.754374 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":74,"threads_started":1}
I20260810 06:35:58.754392 3807 tablet_service.cc:1467] Processing CreateTablet for tablet 595970af904349c2a18e0f9b2b5d2431 (DEFAULT_TABLE table=test-table [id=e485c24d2e2a42088766127b47d7febb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:58.754392 3946 tablet_service.cc:1467] Processing CreateTablet for tablet 595970af904349c2a18e0f9b2b5d2431 (DEFAULT_TABLE table=test-table [id=e485c24d2e2a42088766127b47d7febb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:58.754477 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.751562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.30.71.1:40717)
0810 06:35:58.752671 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.752675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.752687 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:35:58.752717 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.752719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.752736 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.752843 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.754102 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.754109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.754109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.754114 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.754167 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.754188 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.754289 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.754367 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.754385 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":112,"threads_started":1}
I20260810 06:35:58.754792 3946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 595970af904349c2a18e0f9b2b5d2431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:58.754792 3807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 595970af904349c2a18e0f9b2b5d2431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:58.758671 4030 tablet_bootstrap.cc:492] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Bootstrap starting.
I20260810 06:35:58.759094 4031 tablet_bootstrap.cc:492] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Bootstrap starting.
I20260810 06:35:58.759328 3671 tablet_service.cc:1467] Processing CreateTablet for tablet 595970af904349c2a18e0f9b2b5d2431 (DEFAULT_TABLE table=test-table [id=e485c24d2e2a42088766127b47d7febb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:35:58.759698 3671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 595970af904349c2a18e0f9b2b5d2431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:58.759752 4030 tablet_bootstrap.cc:654] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:58.759985 4031 tablet_bootstrap.cc:654] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:58.760257 4030 log.cc:824] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:58.760373 4031 log.cc:824] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:58.761173 4030 tablet_bootstrap.cc:492] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: No bootstrap required, opened a new log
I20260810 06:35:58.761277 4030 ts_tablet_manager.cc:1397] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 06:35:58.761279 4031 tablet_bootstrap.cc:492] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: No bootstrap required, opened a new log
I20260810 06:35:58.761353 4031 ts_tablet_manager.cc:1397] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:35:58.763597 4034 tablet_bootstrap.cc:492] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d: Bootstrap starting.
I20260810 06:35:58.764189 4030 raft_consensus.cc:348] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.764297 4031 raft_consensus.cc:348] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.764405 4030 raft_consensus.cc:374] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:58.764477 4031 raft_consensus.cc:374] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:58.764490 4030 raft_consensus.cc:729] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b89aac0303bc45a3927d9adb8ddcf0ec, State: Initialized, Role: FOLLOWER
I20260810 06:35:58.764514 4031 raft_consensus.cc:729] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8511e161102f45d0a9a5f8c958717e60, State: Initialized, Role: FOLLOWER
I20260810 06:35:58.764540 4034 tablet_bootstrap.cc:654] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:58.764631 4030 consensus_queue.cc:260] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.764662 4031 consensus_queue.cc:260] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.764928 4034 log.cc:824] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:58.764977 4030 ts_tablet_manager.cc:1428] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260810 06:35:58.764997 4014 heartbeater.cc:503] Master 127.30.71.62:39753 was elected leader, sending a full tablet report...
I20260810 06:35:58.764978 4031 ts_tablet_manager.cc:1428] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 06:35:58.765105 3876 heartbeater.cc:503] Master 127.30.71.62:39753 was elected leader, sending a full tablet report...
I20260810 06:35:58.765756 4034 tablet_bootstrap.cc:492] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d: No bootstrap required, opened a new log
I20260810 06:35:58.765857 4034 ts_tablet_manager.cc:1397] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:35:58.768254 4034 raft_consensus.cc:348] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.768433 4034 raft_consensus.cc:374] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:58.768487 4034 raft_consensus.cc:729] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee193bcfe36a4026bedc4d5b51398f3d, State: Initialized, Role: FOLLOWER
I20260810 06:35:58.768633 4034 consensus_queue.cc:260] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.768911 4034 ts_tablet_manager.cc:1428] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:35:58.768975 3740 heartbeater.cc:503] Master 127.30.71.62:39753 was elected leader, sending a full tablet report...
I20260810 06:35:58.804082 4036 raft_consensus.cc:482] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:35:58.804248 4036 raft_consensus.cc:504] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.804694 4036 leader_election.cc:288] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8511e161102f45d0a9a5f8c958717e60 (127.30.71.3:38033), ee193bcfe36a4026bedc4d5b51398f3d (127.30.71.1:40717)
I20260810 06:35:58.806145 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.804896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38033 (local address 127.30.71.2:52497)
0810 06:35:58.804956 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.804970 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.805011 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.805185 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.805186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.805189 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.805768 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.805769 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.805777 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.805788 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.805975 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.805975 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.805976 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.806044 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.806059 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:35:58.806173 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.804978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:52497 (local address 127.30.71.3:38033)
0810 06:35:58.805051 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.805054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.805065 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:35:58.805097 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.805098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.805105 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.805173 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.805875 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.805876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.805877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.805882 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.805899 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.805904 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.805977 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.806064 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.806079 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:35:58.806255 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.805028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:40717 (local address 127.30.71.2:40029)
0810 06:35:58.805238 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.805247 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:58.805316 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:58.805490 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:58.805490 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:58.805493 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:35:58.805932 (+ 439us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:35:58.805933 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:35:58.805940 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:35:58.805953 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:35:58.806116 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:35:58.806116 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:35:58.806117 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:35:58.806160 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 06:35:58.806172 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":129,"threads_started":1}
I20260810 06:35:58.806303 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.805196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:40029 (local address 127.30.71.1:40717)
0810 06:35:58.805301 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.805304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.805314 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:35:58.805399 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.805400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.805405 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.805457 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.806038 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.806039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.806040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.806045 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.806061 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.806066 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.806117 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.806168 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.806182 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:35:58.806500 3966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595970af904349c2a18e0f9b2b5d2431" candidate_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8511e161102f45d0a9a5f8c958717e60" is_pre_election: true
I20260810 06:35:58.806627 3966 raft_consensus.cc:2454] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b89aac0303bc45a3927d9adb8ddcf0ec in term 0.
I20260810 06:35:58.806881 3761 leader_election.cc:302] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8511e161102f45d0a9a5f8c958717e60, b89aac0303bc45a3927d9adb8ddcf0ec; no voters:
I20260810 06:35:58.807044 3692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595970af904349c2a18e0f9b2b5d2431" candidate_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" is_pre_election: true
I20260810 06:35:58.807109 4036 raft_consensus.cc:2781] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:35:58.807160 4036 raft_consensus.cc:482] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:35:58.807195 4036 raft_consensus.cc:3037] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:58.807219 3692 raft_consensus.cc:2454] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b89aac0303bc45a3927d9adb8ddcf0ec in term 0.
I20260810 06:35:58.808012 4036 raft_consensus.cc:504] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.808146 4036 leader_election.cc:288] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [CANDIDATE]: Term 1 election: Requested vote from peers 8511e161102f45d0a9a5f8c958717e60 (127.30.71.3:38033), ee193bcfe36a4026bedc4d5b51398f3d (127.30.71.1:40717)
I20260810 06:35:58.808336 3966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595970af904349c2a18e0f9b2b5d2431" candidate_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8511e161102f45d0a9a5f8c958717e60"
I20260810 06:35:58.808352 3692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595970af904349c2a18e0f9b2b5d2431" candidate_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee193bcfe36a4026bedc4d5b51398f3d"
I20260810 06:35:58.808404 3966 raft_consensus.cc:3037] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:58.808436 3692 raft_consensus.cc:3037] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:58.809125 3966 raft_consensus.cc:2454] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b89aac0303bc45a3927d9adb8ddcf0ec in term 1.
I20260810 06:35:58.809211 3692 raft_consensus.cc:2454] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b89aac0303bc45a3927d9adb8ddcf0ec in term 1.
I20260810 06:35:58.809350 3761 leader_election.cc:302] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8511e161102f45d0a9a5f8c958717e60, b89aac0303bc45a3927d9adb8ddcf0ec; no voters:
I20260810 06:35:58.809492 4036 raft_consensus.cc:2781] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:58.809643 4036 raft_consensus.cc:686] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 1 LEADER]: Becoming Leader. State: Replica: b89aac0303bc45a3927d9adb8ddcf0ec, State: Running, Role: LEADER
I20260810 06:35:58.809773 4036 consensus_queue.cc:237] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } }
I20260810 06:35:58.810671 3541 catalog_manager.cc:5495] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec reported cstate change: term changed from 0 to 1, leader changed from <none> to b89aac0303bc45a3927d9adb8ddcf0ec (127.30.71.2). New cstate: current_term: 1 leader_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b89aac0303bc45a3927d9adb8ddcf0ec" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 } health_report { overall_health: UNKNOWN } } }
I20260810 06:35:58.819204 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.818163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.30.71.2:34103)
0810 06:35:58.818276 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:58.818279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:58.818287 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:35:58.818324 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:58.818325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:58.818346 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:58.818419 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:58.818950 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:58.818952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:58.818952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:58.818957 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:58.818992 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:58.818997 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:58.819055 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:58.819104 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 06:35:58.819119 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 06:35:58.821065 3692 raft_consensus.cc:1261] T 595970af904349c2a18e0f9b2b5d2431 P ee193bcfe36a4026bedc4d5b51398f3d [term 1 FOLLOWER]: Refusing update from remote peer b89aac0303bc45a3927d9adb8ddcf0ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:58.821081 3966 raft_consensus.cc:1261] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 1 FOLLOWER]: Refusing update from remote peer b89aac0303bc45a3927d9adb8ddcf0ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:35:58.821434 4036 consensus_queue.cc:1035] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee193bcfe36a4026bedc4d5b51398f3d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 40717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:58.821573 4040 consensus_queue.cc:1035] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "8511e161102f45d0a9a5f8c958717e60" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:35:58.824491 4044 mvcc.cc:205] Tried to move back new op lower bound from 7316864036128309248 to 7316864036085673984. Current Snapshot: MvccSnapshot[applied={T|T < 7316864036128309248}]
I20260810 06:35:58.824826 4043 mvcc.cc:205] Tried to move back new op lower bound from 7316864036128309248 to 7316864036085673984. Current Snapshot: MvccSnapshot[applied={T|T < 7316864036128309248 or (T in {7316864036128309248})}]
I20260810 06:35:58.825035 4045 mvcc.cc:205] Tried to move back new op lower bound from 7316864036128309248 to 7316864036085673984. Current Snapshot: MvccSnapshot[applied={T|T < 7316864036128309248}]
I20260810 06:35:58.826638 3541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:58.826753 3541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:35:58.827561 3541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d4a2666e13e43cb9d40df9ac3550735: Sending DeleteTablet for 3 replicas of tablet 595970af904349c2a18e0f9b2b5d2431
I20260810 06:35:58.827963 3671 tablet_service.cc:1514] Processing DeleteTablet for tablet 595970af904349c2a18e0f9b2b5d2431 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:58 UTC) from {username='slave'} at 127.0.0.1:51910
I20260810 06:35:58.827971 3946 tablet_service.cc:1514] Processing DeleteTablet for tablet 595970af904349c2a18e0f9b2b5d2431 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:58 UTC) from {username='slave'} at 127.0.0.1:39606
I20260810 06:35:58.828148 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 3606
I20260810 06:35:58.828190 3807 tablet_service.cc:1514] Processing DeleteTablet for tablet 595970af904349c2a18e0f9b2b5d2431 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:35:58 UTC) from {username='slave'} at 127.0.0.1:54972
I20260810 06:35:58.828258 4053 tablet_replica.cc:330] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: stopping tablet replica
I20260810 06:35:58.828351 4053 raft_consensus.cc:2229] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:35:58.828444 4053 raft_consensus.cc:2258] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:58.829018 4053 ts_tablet_manager.cc:1905] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:58.830933 4053 ts_tablet_manager.cc:1918] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:35:58.831043 4053 log.cc:1196] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/595970af904349c2a18e0f9b2b5d2431
I20260810 06:35:58.831343 4053 ts_tablet_manager.cc:1939] T 595970af904349c2a18e0f9b2b5d2431 P 8511e161102f45d0a9a5f8c958717e60: Deleting consensus metadata
I20260810 06:35:58.831792 3529 catalog_manager.cc:4841] TS 8511e161102f45d0a9a5f8c958717e60 (127.30.71.3:38033): tablet 595970af904349c2a18e0f9b2b5d2431 (table test-table [id=e485c24d2e2a42088766127b47d7febb]) successfully deleted
I20260810 06:35:58.834131 4055 tablet_replica.cc:330] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: stopping tablet replica
I20260810 06:35:58.834275 4055 raft_consensus.cc:2229] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:35:58.834394 4055 raft_consensus.cc:2258] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:35:58.834875 4055 ts_tablet_manager.cc:1905] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:35:58.836978 4055 ts_tablet_manager.cc:1918] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:35:58.837060 4055 log.cc:1196] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/595970af904349c2a18e0f9b2b5d2431
I20260810 06:35:58.837363 4055 ts_tablet_manager.cc:1939] T 595970af904349c2a18e0f9b2b5d2431 P b89aac0303bc45a3927d9adb8ddcf0ec: Deleting consensus metadata
I20260810 06:35:58.837783 3529 catalog_manager.cc:4841] TS b89aac0303bc45a3927d9adb8ddcf0ec (127.30.71.2:34103): tablet 595970af904349c2a18e0f9b2b5d2431 (table test-table [id=e485c24d2e2a42088766127b47d7febb]) successfully deleted
W20260810 06:35:58.838678 3529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:40717 (error 108)
I20260810 06:35:58.839026 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 3743
I20260810 06:35:58.841717 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:58.839183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:40717 (local address 127.0.0.1:51912)
0810 06:35:58.841633 (+ 2450us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:58.841680 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2427}
W20260810 06:35:58.841851 3529 catalog_manager.cc:4584] TS ee193bcfe36a4026bedc4d5b51398f3d (127.30.71.1:40717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 595970af904349c2a18e0f9b2b5d2431: Network error: Client connection negotiation failed: client connection to 127.30.71.1:40717: connect: Connection refused (error 111)
I20260810 06:35:58.848501 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 3879
I20260810 06:35:58.855816 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 3509
2026-08-10T06:35:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (669 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 06:35:58.877041 31004 mini_oidc.cc:152] Starting JWKS server
I20260810 06:35:58.878233 31004 webserver.cc:460] Webserver started at https://127.0.0.1:38549/ using document root <none> and password file <none>
I20260810 06:35:58.878361 31004 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:35:58.878546 31004 webserver.cc:460] Webserver started at http://127.0.0.1:44195/ using document root <none> and password file <none>
2026-08-10T06:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:58Z Disabled control of system clock
I20260810 06:35:58.887161 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37939
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:43917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:37939
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38549/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/testchainca.cert with env {}
W20260810 06:35:58.970019 4062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:58.970253 4062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:58.970278 4062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:58.971582 4062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:58.971634 4062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:58.971648 4062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:58.971660 4062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:58.971673 4062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:35:58.971685 4062 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:35:58.971731 4062 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38549/jwks.json
I20260810 06:35:58.973472 4062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:37939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38549/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:58.973778 4062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:58.974037 4062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:58.979152 4069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.979319 4068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:58.979152 4071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:58.979372 4062 server_base.cc:1034] running on GCE node
I20260810 06:35:58.979612 4062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:58.979954 4062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:58.981127 4062 hybrid_clock.cc:648] HybridClock initialized: now 1786343758981114 us; error 37 us; skew 500 ppm
I20260810 06:35:58.982442 4062 webserver.cc:460] Webserver started at http://127.30.71.62:42935/ using document root <none> and password file <none>
I20260810 06:35:58.982707 4062 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:58.982796 4062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:58.982955 4062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:58.984201 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "4061d78d49e4406f909bba780fc01286"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "dc29c4f06f746b0fafe842bce5e1d220"
server_key_iv: "db8178450285b43891a1329ccd01e4fc"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.984623 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "4061d78d49e4406f909bba780fc01286"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "dc29c4f06f746b0fafe842bce5e1d220"
server_key_iv: "db8178450285b43891a1329ccd01e4fc"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.986310 4062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:35:58.987195 4077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:58.987375 4062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:58.987491 4062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "4061d78d49e4406f909bba780fc01286"
format_stamp: "Formatted at 2026-08-10 06:35:58 on dist-test-slave-w206"
server_key: "dc29c4f06f746b0fafe842bce5e1d220"
server_key_iv: "db8178450285b43891a1329ccd01e4fc"
server_key_version: "encryptionkey@0"
I20260810 06:35:58.987572 4062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:58.998822 4062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:58.999212 4062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:58.999384 4062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:59.003496 4062 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:37939
I20260810 06:35:59.003532 4130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:37939 every 8 connection(s)
I20260810 06:35:59.003891 4062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:59.004936 4131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:59.007412 4131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286: Bootstrap starting.
I20260810 06:35:59.008108 4131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:59.008426 4131 log.cc:824] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:59.009161 4131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286: No bootstrap required, opened a new log
I20260810 06:35:59.010792 4131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4061d78d49e4406f909bba780fc01286" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37939 } }
I20260810 06:35:59.010939 4131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:59.010962 4131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4061d78d49e4406f909bba780fc01286, State: Initialized, Role: FOLLOWER
I20260810 06:35:59.011188 4131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4061d78d49e4406f909bba780fc01286" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37939 } }
I20260810 06:35:59.011279 4131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:59.011322 4131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:59.011373 4131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:59.011618 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4062
I20260810 06:35:59.011727 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:59.011920 31004 external_mini_cluster.cc:1428] Setting key f603eeda455e412585c26896cfcbf80a
I20260810 06:35:59.012151 4131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4061d78d49e4406f909bba780fc01286" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37939 } }
I20260810 06:35:59.012286 4131 leader_election.cc:302] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4061d78d49e4406f909bba780fc01286; no voters:
I20260810 06:35:59.012491 4131 leader_election.cc:288] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:59.012683 4136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:59.012802 4131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:59.012873 4136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [term 1 LEADER]: Becoming Leader. State: Replica: 4061d78d49e4406f909bba780fc01286, State: Running, Role: LEADER
I20260810 06:35:59.012964 4136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4061d78d49e4406f909bba780fc01286" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37939 } }
I20260810 06:35:59.013468 4135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4061d78d49e4406f909bba780fc01286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4061d78d49e4406f909bba780fc01286" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37939 } } }
I20260810 06:35:59.013568 4135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.013738 4139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4061d78d49e4406f909bba780fc01286. Latest consensus state: current_term: 1 leader_uuid: "4061d78d49e4406f909bba780fc01286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4061d78d49e4406f909bba780fc01286" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37939 } } }
I20260810 06:35:59.013814 4139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.014206 4146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:59.014938 4146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:59.016920 4146 catalog_manager.cc:1347] Generated new cluster ID: 6701229c13204fac9543d02a1483142d
I20260810 06:35:59.016978 4146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:59.019878 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.012515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.30.71.62:37939)
0810 06:35:59.012771 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.012797 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.012830 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:35:59.012991 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.012997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.013077 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.013467 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:59.013579 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.014882 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.015808 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.016428 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.019111 (+ 2683us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.019247 (+ 136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:59.019250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:59.019250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:59.019420 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:59.019447 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:59.019456 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:59.019535 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:59.019615 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 06:35:59.019696 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":121,"threads_started":1}
I20260810 06:35:59.020824 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:35:59.042369 4146 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:59.043159 4146 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:59.048506 4146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4061d78d49e4406f909bba780fc01286: Generated new TSK 0
I20260810 06:35:59.048727 4146 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:35:59.081787 4163 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 06:35:59.081946 4163 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260810 06:35:59.082610 4162 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:35:59.065751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37939 (local address 127.0.0.1:60636)
0810 06:35:59.065971 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:59.065980 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:59.066048 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:59.066288 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:59.066289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:59.066291 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:35:59.066354 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.066356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.067338 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.067339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.067957 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.067959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.068464 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.068464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.068513 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.068533 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:35:59.082068 (+ 13535us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38549/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:35:59.082183 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.71.62:37939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38549/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
I20260810 06:35:59.082924 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.065870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.30.71.62:37939)
0810 06:35:59.065938 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.065940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.066046 (+ 106us) server_negotiation.cc:407] Connection header received
0810 06:35:59.066136 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.066137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.066150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.066233 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:35:59.066434 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.067226 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.068044 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.068368 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.068440 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.068647 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:35:59.081897 (+ 13250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38549/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:35:59.082858 (+ 961us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60636: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38549/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 06:35:59.083002 4137 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60636: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38549/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.71.62:37939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38549/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 06:35:59.083182 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4062
2026-08-10T06:35:59Z chronyd exiting
I20260810 06:35:59.121172 31004 test_util.cc:175] -----------------------------------------------
I20260810 06:35:59.121258 31004 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786343734291041-31004-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (245 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 06:35:59.122146 31004 mini_oidc.cc:152] Starting JWKS server
I20260810 06:35:59.122862 31004 webserver.cc:460] Webserver started at https://127.0.0.1:44945/ using document root <none> and password file <none>
I20260810 06:35:59.122932 31004 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:35:59.123136 31004 webserver.cc:460] Webserver started at http://127.0.0.1:38243/ using document root <none> and password file <none>
2026-08-10T06:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:59Z Disabled control of system clock
I20260810 06:35:59.131456 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41497
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:45007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:41497
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44945/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/testchainca.cert with env {}
W20260810 06:35:59.213354 4169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:59.213635 4169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:59.213675 4169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:59.214905 4169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:59.214982 4169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:59.215027 4169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:59.215059 4169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:59.215081 4169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:35:59.215106 4169 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:35:59.215137 4169 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44945/jwks.json
I20260810 06:35:59.216746 4169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:45007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:41497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44945/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:59.217049 4169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:59.217291 4169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:59.222271 4175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:59.222417 4169 server_base.cc:1034] running on GCE node
W20260810 06:35:59.222283 4178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:59.222308 4176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:59.222767 4169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:59.223171 4169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:59.224355 4169 hybrid_clock.cc:648] HybridClock initialized: now 1786343759224328 us; error 40 us; skew 500 ppm
I20260810 06:35:59.225741 4169 webserver.cc:460] Webserver started at http://127.30.71.62:44903/ using document root <none> and password file <none>
I20260810 06:35:59.226014 4169 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:59.226089 4169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:59.226243 4169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:59.227526 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "845dbfd9e62d48638624275f159e449d"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "a1a2d7a667493d55ad605260e05bb60f"
server_key_iv: "842cf2e19e2fdfb61398c3c21936b322"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.227954 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "845dbfd9e62d48638624275f159e449d"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "a1a2d7a667493d55ad605260e05bb60f"
server_key_iv: "842cf2e19e2fdfb61398c3c21936b322"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.229538 4169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:35:59.230356 4185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:59.230540 4169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:59.230650 4169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "845dbfd9e62d48638624275f159e449d"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "a1a2d7a667493d55ad605260e05bb60f"
server_key_iv: "842cf2e19e2fdfb61398c3c21936b322"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.230727 4169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:59.240659 4169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:59.240975 4169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:59.241132 4169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:59.245204 4169 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:41497
I20260810 06:35:59.245235 4242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:41497 every 8 connection(s)
I20260810 06:35:59.245606 4169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:59.246117 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4169
I20260810 06:35:59.246224 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:59.246467 31004 external_mini_cluster.cc:1428] Setting key 8b88fd8c4d63177f874a784aca719c25
I20260810 06:35:59.246475 4243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:59.249751 4243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d: Bootstrap starting.
I20260810 06:35:59.250433 4243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:59.250752 4243 log.cc:824] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:59.251508 4243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d: No bootstrap required, opened a new log
I20260810 06:35:59.252493 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.247110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35932 (local address 127.30.71.62:41497)
0810 06:35:59.247348 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.247356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.247386 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:35:59.248070 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.248078 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.248141 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.248304 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:59.248527 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.249403 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.250306 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.250800 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.250938 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.251725 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:59.251729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:59.251729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:59.251956 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:59.252002 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:59.252018 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:59.252105 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:59.252171 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 06:35:59.252261 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260810 06:35:59.253634 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:35:59.253901 4243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845dbfd9e62d48638624275f159e449d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41497 } }
I20260810 06:35:59.254076 4243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:59.254117 4243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 845dbfd9e62d48638624275f159e449d, State: Initialized, Role: FOLLOWER
I20260810 06:35:59.254244 4243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845dbfd9e62d48638624275f159e449d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41497 } }
I20260810 06:35:59.254321 4243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:59.254354 4243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:59.254407 4243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:59.255179 4243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845dbfd9e62d48638624275f159e449d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41497 } }
I20260810 06:35:59.255336 4243 leader_election.cc:302] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 845dbfd9e62d48638624275f159e449d; no voters:
I20260810 06:35:59.255532 4243 leader_election.cc:288] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:59.255605 4249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:59.255776 4249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [term 1 LEADER]: Becoming Leader. State: Replica: 845dbfd9e62d48638624275f159e449d, State: Running, Role: LEADER
I20260810 06:35:59.255795 4243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:59.255872 4249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845dbfd9e62d48638624275f159e449d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41497 } }
I20260810 06:35:59.256234 4250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "845dbfd9e62d48638624275f159e449d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845dbfd9e62d48638624275f159e449d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41497 } } }
I20260810 06:35:59.256462 4251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 845dbfd9e62d48638624275f159e449d. Latest consensus state: current_term: 1 leader_uuid: "845dbfd9e62d48638624275f159e449d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845dbfd9e62d48638624275f159e449d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41497 } } }
I20260810 06:35:59.256538 4250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.256556 4251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.256901 4259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:59.257444 4259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:59.259037 4259 catalog_manager.cc:1347] Generated new cluster ID: 58f92185e3384429b1b690967976af14
I20260810 06:35:59.259095 4259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:59.271373 4259 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:59.271932 4259 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:59.278051 4259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 845dbfd9e62d48638624275f159e449d: Generated new TSK 0
I20260810 06:35:59.278272 4259 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:35:59.309067 4275 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 06:35:59.309163 4275 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:35:59.309479 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.283495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.30.71.62:41497)
0810 06:35:59.283563 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.283566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.287195 (+ 3629us) server_negotiation.cc:407] Connection header received
0810 06:35:59.287280 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.287281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.287296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.287380 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:35:59.287536 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.288071 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.289219 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.289428 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.289487 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.289686 (+ 199us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:35:59.309262 (+ 19576us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:35:59.309422 (+ 160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 06:35:59.309558 4246 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35946: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 06:35:59.309648 4273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:35:59.283430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41497 (local address 127.0.0.1:35946)
0810 06:35:59.287104 (+ 3674us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:59.287117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:59.287197 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:59.287393 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:59.287394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:59.287396 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:35:59.287465 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.287468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.288154 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.288155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.289118 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.289122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.289511 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.289512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.289577 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.289584 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:35:59.309408 (+ 19824us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:35:59.309551 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.71.62:41497: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":3634,"thread_start_us":90,"threads_started":1}
I20260810 06:35:59.310003 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4169
2026-08-10T06:35:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 06:35:59.350018 31004 mini_oidc.cc:152] Starting JWKS server
I20260810 06:35:59.350759 31004 webserver.cc:460] Webserver started at https://127.0.0.1:33211/ using document root <none> and password file <none>
I20260810 06:35:59.350821 31004 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:35:59.351011 31004 webserver.cc:460] Webserver started at http://127.0.0.1:35983/ using document root <none> and password file <none>
2026-08-10T06:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:59Z Disabled control of system clock
I20260810 06:35:59.360790 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:38877
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:38877
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33211/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:35:59.443632 4281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:59.443895 4281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:59.443934 4281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:59.445148 4281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:59.445223 4281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:59.445245 4281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:59.445348 4281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:59.445377 4281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:35:59.445408 4281 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:35:59.445437 4281 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33211/jwks.json
I20260810 06:35:59.446894 4281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:38877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:38877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33211/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:59.447209 4281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:59.447522 4281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:59.452598 4287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:59.452636 4290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:59.452610 4286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:59.452736 4281 server_base.cc:1034] running on GCE node
I20260810 06:35:59.453033 4281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:59.453289 4281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:59.454435 4281 hybrid_clock.cc:648] HybridClock initialized: now 1786343759454422 us; error 33 us; skew 500 ppm
I20260810 06:35:59.455827 4281 webserver.cc:460] Webserver started at http://127.30.71.62:34779/ using document root <none> and password file <none>
I20260810 06:35:59.456099 4281 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:59.456151 4281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:59.456319 4281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:59.457545 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "fe6f2a0ab52142a3ac988d59c6d5494a"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "86a2183ebd826e81571d0bfdd18e731e"
server_key_iv: "26b622bc27d7cdcde78f88e33e324a5b"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.457944 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "fe6f2a0ab52142a3ac988d59c6d5494a"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "86a2183ebd826e81571d0bfdd18e731e"
server_key_iv: "26b622bc27d7cdcde78f88e33e324a5b"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.459542 4281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:35:59.460449 4296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:59.460669 4281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:35:59.460747 4281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "fe6f2a0ab52142a3ac988d59c6d5494a"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "86a2183ebd826e81571d0bfdd18e731e"
server_key_iv: "26b622bc27d7cdcde78f88e33e324a5b"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.460847 4281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:59.472134 4281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:59.472484 4281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:59.472635 4281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:59.477196 4281 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:38877
I20260810 06:35:59.477237 4350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:38877 every 8 connection(s)
I20260810 06:35:59.477643 4281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:59.478328 4351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:59.481022 4351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a: Bootstrap starting.
I20260810 06:35:59.481644 4351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:59.481904 4351 log.cc:824] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:59.482589 4351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a: No bootstrap required, opened a new log
I20260810 06:35:59.484300 4351 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38877 } }
I20260810 06:35:59.484454 4351 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:59.484477 4351 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6f2a0ab52142a3ac988d59c6d5494a, State: Initialized, Role: FOLLOWER
I20260810 06:35:59.484632 4351 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38877 } }
I20260810 06:35:59.484721 4351 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:59.484745 4351 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:59.484787 4351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:59.485327 4351 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38877 } }
I20260810 06:35:59.485416 4351 leader_election.cc:302] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe6f2a0ab52142a3ac988d59c6d5494a; no voters:
I20260810 06:35:59.485613 4351 leader_election.cc:288] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:59.485688 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4281
I20260810 06:35:59.485684 4355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:59.485786 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:59.485852 4355 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [term 1 LEADER]: Becoming Leader. State: Replica: fe6f2a0ab52142a3ac988d59c6d5494a, State: Running, Role: LEADER
I20260810 06:35:59.485932 4351 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:59.485980 31004 external_mini_cluster.cc:1428] Setting key ac88321497a844ab7d3721d7fba45934
I20260810 06:35:59.485957 4355 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38877 } }
I20260810 06:35:59.486382 4356 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38877 } } }
I20260810 06:35:59.486477 4356 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.486706 4357 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe6f2a0ab52142a3ac988d59c6d5494a. Latest consensus state: current_term: 1 leader_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6f2a0ab52142a3ac988d59c6d5494a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 38877 } } }
I20260810 06:35:59.487134 4357 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.487052 4363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:59.487784 4363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:59.489590 4363 catalog_manager.cc:1347] Generated new cluster ID: 716000f2f8534065bb9bcb2331581167
I20260810 06:35:59.489652 4363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:59.492542 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.486640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35216 (local address 127.30.71.62:38877)
0810 06:35:59.487169 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.487179 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.487217 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:35:59.487345 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.487353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.487426 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.487571 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:59.488032 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.489214 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.490326 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.490945 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.491095 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.491830 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:59.491833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:59.491834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:59.492034 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:59.492074 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:59.492087 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:59.492152 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:59.492256 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 06:35:59.492340 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":69,"threads_started":1}
I20260810 06:35:59.493404 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:35:59.501960 4363 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:59.502668 4363 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:59.509258 4363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe6f2a0ab52142a3ac988d59c6d5494a: Generated new TSK 0
I20260810 06:35:59.509467 4363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:35:59.518224 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.514821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35224 (local address 127.30.71.62:38877)
0810 06:35:59.514893 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.514896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.515047 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:35:59.515131 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.515132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.515144 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.515216 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:35:59.515355 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.515866 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.516676 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.516906 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.516982 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.517187 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:35:59.517995 (+ 808us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:35:59.518149 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 06:35:59.518288 4383 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:35:59.514747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:38877 (local address 127.0.0.1:35224)
0810 06:35:59.514946 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:59.514959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:59.515066 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:59.515218 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:59.515219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:59.515221 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:35:59.515287 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.515289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.515953 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.515954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.516574 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.516576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.516989 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.516991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.517052 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.517058 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:35:59.518099 (+ 1041us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:35:59.518193 (+ 94us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.71.62:38877: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":114,"threads_started":1}
W20260810 06:35:59.518306 4366 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260810 06:35:59.518524 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4281
E20260810 06:35:59.533457 4384 webserver.cc:533] Webserver: sslize failed:
E20260810 06:35:59.533545 4384 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T06:35:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T06:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:35:59Z Disabled control of system clock
I20260810 06:35:59.570094 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:34847
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:34847
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/testchainca.cert with env {}
W20260810 06:35:59.653326 4388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:35:59.653589 4388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:35:59.653630 4388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:35:59.654852 4388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:35:59.654937 4388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:35:59.654961 4388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:35:59.654992 4388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:35:59.655081 4388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:35:59.655117 4388 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:35:59.655148 4388 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 06:35:59.656716 4388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:34847
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:34847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:35:59.656999 4388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:35:59.657249 4388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:35:59.662408 4397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:59.662393 4395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:35:59.662375 4393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:35:59.662644 4388 server_base.cc:1034] running on GCE node
I20260810 06:35:59.662885 4388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:35:59.663208 4388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:35:59.664390 4388 hybrid_clock.cc:648] HybridClock initialized: now 1786343759664368 us; error 41 us; skew 500 ppm
I20260810 06:35:59.665699 4388 webserver.cc:460] Webserver started at http://127.30.71.62:34691/ using document root <none> and password file <none>
I20260810 06:35:59.665970 4388 fs_manager.cc:359] Metadata directory not provided
I20260810 06:35:59.666045 4388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:35:59.666204 4388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:35:59.667497 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "f6a74705932c4bfb9c5416fdf6b18523"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "a4a4d957aa8b343ac3c882318de9c5e9"
server_key_iv: "cab1daaa353281019c0802a4bea989ac"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.667968 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "f6a74705932c4bfb9c5416fdf6b18523"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "a4a4d957aa8b343ac3c882318de9c5e9"
server_key_iv: "cab1daaa353281019c0802a4bea989ac"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.669479 4388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:35:59.670351 4404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:35:59.670531 4388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:35:59.670640 4388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "f6a74705932c4bfb9c5416fdf6b18523"
format_stamp: "Formatted at 2026-08-10 06:35:59 on dist-test-slave-w206"
server_key: "a4a4d957aa8b343ac3c882318de9c5e9"
server_key_iv: "cab1daaa353281019c0802a4bea989ac"
server_key_version: "encryptionkey@0"
I20260810 06:35:59.670725 4388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:35:59.680537 4388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:35:59.680853 4388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:35:59.681007 4388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:35:59.684830 4388 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:34847
I20260810 06:35:59.684899 4458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:34847 every 8 connection(s)
I20260810 06:35:59.685264 4388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:35:59.686267 4459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:35:59.688623 4459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523: Bootstrap starting.
I20260810 06:35:59.689280 4459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523: Neither blocks nor log segments found. Creating new log.
I20260810 06:35:59.689569 4459 log.cc:824] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523: Log is configured to *not* fsync() on all Append() calls
I20260810 06:35:59.690280 4459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523: No bootstrap required, opened a new log
I20260810 06:35:59.692010 4459 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a74705932c4bfb9c5416fdf6b18523" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 34847 } }
I20260810 06:35:59.692150 4459 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:35:59.692173 4459 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a74705932c4bfb9c5416fdf6b18523, State: Initialized, Role: FOLLOWER
I20260810 06:35:59.692375 4459 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a74705932c4bfb9c5416fdf6b18523" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 34847 } }
I20260810 06:35:59.692461 4459 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:35:59.692502 4459 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:35:59.692550 4459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:35:59.693089 4459 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a74705932c4bfb9c5416fdf6b18523" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 34847 } }
I20260810 06:35:59.693204 4459 leader_election.cc:302] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a74705932c4bfb9c5416fdf6b18523; no voters:
I20260810 06:35:59.693396 4459 leader_election.cc:288] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:35:59.693436 4462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:35:59.693632 4462 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [term 1 LEADER]: Becoming Leader. State: Replica: f6a74705932c4bfb9c5416fdf6b18523, State: Running, Role: LEADER
I20260810 06:35:59.693661 4459 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:35:59.693766 4462 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a74705932c4bfb9c5416fdf6b18523" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 34847 } }
I20260810 06:35:59.694195 4463 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6a74705932c4bfb9c5416fdf6b18523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a74705932c4bfb9c5416fdf6b18523" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 34847 } } }
I20260810 06:35:59.694346 4463 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.694321 4464 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a74705932c4bfb9c5416fdf6b18523. Latest consensus state: current_term: 1 leader_uuid: "f6a74705932c4bfb9c5416fdf6b18523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a74705932c4bfb9c5416fdf6b18523" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 34847 } } }
I20260810 06:35:59.694478 4464 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523 [sys.catalog]: This master's current role is: LEADER
I20260810 06:35:59.694923 4471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:35:59.695026 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4388
I20260810 06:35:59.695140 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:35:59.695358 31004 external_mini_cluster.cc:1428] Setting key 8e8ef37d80a11e10e9e2a81ba7c3efc3
I20260810 06:35:59.695490 4471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:35:59.699421 4471 catalog_manager.cc:1347] Generated new cluster ID: 4f3d58c7a6e54f1bbd706a18900a650a
I20260810 06:35:59.699486 4471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:35:59.701140 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.696343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.30.71.62:34847)
0810 06:35:59.696541 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.696556 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.696580 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:35:59.696699 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.696706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.696779 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.696927 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:35:59.697151 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.698026 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.698841 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.699467 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.699642 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.700490 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:35:59.700494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:35:59.700495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:35:59.700678 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:35:59.700714 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:35:59.700725 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:35:59.700798 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:35:59.700884 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 06:35:59.700966 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260810 06:35:59.702270 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:35:59.721935 4471 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:35:59.722481 4471 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:35:59.726493 4471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6a74705932c4bfb9c5416fdf6b18523: Generated new TSK 0
I20260810 06:35:59.726706 4471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:35:59.796305 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:35:59.733874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.30.71.62:34847)
0810 06:35:59.733918 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:35:59.733920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:35:59.734012 (+ 92us) server_negotiation.cc:407] Connection header received
0810 06:35:59.734077 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:35:59.734078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:35:59.734089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:35:59.734160 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:35:59.734322 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.734836 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.736018 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.736227 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.736276 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.736468 (+ 192us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:35:59.796049 (+ 59581us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:35:59.796243 (+ 194us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49190: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 06:35:59.796403 4481 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49190: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 06:35:59.796496 4490 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:35:59.733749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:34847 (local address 127.0.0.1:49190)
0810 06:35:59.733921 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:35:59.733932 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:35:59.734006 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:35:59.734181 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:35:59.734181 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:35:59.734184 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:35:59.734254 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.734256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.734919 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.734920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.735919 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:35:59.735922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:35:59.736306 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:35:59.736307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:35:59.736369 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:35:59.736374 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:35:59.796228 (+ 59854us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:35:59.796386 (+ 158us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.71.62:34847: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260810 06:35:59.796813 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4388
2026-08-10T06:35:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (255 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:36:00 dist-test-slave-w206 krb5kdc[4503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:36:00 dist-test-slave-w206 krb5kdc[4503](info): set up 2 sockets
Aug 10 06:36:00 dist-test-slave-w206 krb5kdc[4503](info): commencing operation
krb5kdc: starting...
W20260810 06:36:01.100703 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:36:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:36:01.230212 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33281
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:33281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:01.314029 4518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:01.314311 4518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:01.314352 4518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:01.315675 4518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:01.315752 4518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:01.315778 4518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:01.315800 4518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:01.315826 4518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:01.317435 4518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:33281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:01.317750 4518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:01.318010 4518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:01.323164 4528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:01.323168 4525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:01.323437 4518 server_base.cc:1034] running on GCE node
W20260810 06:36:01.323469 4524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:01.323762 4518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:01.324064 4518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:01.325237 4518 hybrid_clock.cc:648] HybridClock initialized: now 1786343761325223 us; error 30 us; skew 500 ppm
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.330771 4518 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:36:01.331295 4518 webserver.cc:460] Webserver started at http://127.30.71.62:40507/ using document root <none> and password file <none>
I20260810 06:36:01.331559 4518 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:01.331629 4518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:01.331787 4518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:01.332880 4518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "33558536909744c3b2d1bf31554e9446"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "88c7837a92b181aaa7a73281d6d571af"
server_key_iv: "66341472fbaf4af7eac86a53a6172765"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.333258 4518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "33558536909744c3b2d1bf31554e9446"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "88c7837a92b181aaa7a73281d6d571af"
server_key_iv: "66341472fbaf4af7eac86a53a6172765"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.334709 4518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:01.335515 4536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.335685 4518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.335801 4518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "33558536909744c3b2d1bf31554e9446"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "88c7837a92b181aaa7a73281d6d571af"
server_key_iv: "66341472fbaf4af7eac86a53a6172765"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.335879 4518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:01.354324 4518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:01.356601 4518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:01.356793 4518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:01.361148 4518 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:33281
I20260810 06:36:01.361193 4593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:33281 every 8 connection(s)
I20260810 06:36:01.361573 4518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:01.362555 4594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:01.364899 4594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446: Bootstrap starting.
I20260810 06:36:01.365449 4594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:01.365689 4594 log.cc:824] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:01.365720 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4518
I20260810 06:36:01.365870 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:36:01.366110 31004 external_mini_cluster.cc:1428] Setting key a2eda950b89bab808d8d18abfcff5b85
I20260810 06:36:01.366510 4594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446: No bootstrap required, opened a new log
I20260810 06:36:01.368703 4594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33558536909744c3b2d1bf31554e9446" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33281 } }
I20260810 06:36:01.368892 4594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:01.368927 4594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33558536909744c3b2d1bf31554e9446, State: Initialized, Role: FOLLOWER
I20260810 06:36:01.369076 4594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33558536909744c3b2d1bf31554e9446" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33281 } }
I20260810 06:36:01.369167 4594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:01.369213 4594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:01.369263 4594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:01.369954 4594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33558536909744c3b2d1bf31554e9446" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33281 } }
I20260810 06:36:01.370066 4594 leader_election.cc:302] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33558536909744c3b2d1bf31554e9446; no voters:
I20260810 06:36:01.370223 4594 leader_election.cc:288] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:36:01.370296 4600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:01.370452 4594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:01.370543 4600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [term 1 LEADER]: Becoming Leader. State: Replica: 33558536909744c3b2d1bf31554e9446, State: Running, Role: LEADER
I20260810 06:36:01.370673 4600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33558536909744c3b2d1bf31554e9446" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33281 } }
I20260810 06:36:01.371141 4600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33558536909744c3b2d1bf31554e9446. Latest consensus state: current_term: 1 leader_uuid: "33558536909744c3b2d1bf31554e9446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33558536909744c3b2d1bf31554e9446" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33281 } } }
I20260810 06:36:01.371156 4601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33558536909744c3b2d1bf31554e9446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33558536909744c3b2d1bf31554e9446" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33281 } } }
I20260810 06:36:01.371315 4601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:01.371538 4600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:01.372017 4609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:01.372589 4609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:01.374368 4609 catalog_manager.cc:1347] Generated new cluster ID: c9ea524af6b94b91887c1dbbf84eb58b
I20260810 06:36:01.374435 4609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:01.378772 4609 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:36:01.379578 4609 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:01.379932 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.366666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.30.71.62:33281)
0810 06:36:01.366892 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.366902 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.366939 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:36:01.367896 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.367906 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.367979 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.368137 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.368749 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.370468 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.371750 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.372366 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.372489 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.375505 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:01.375509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:01.375509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:01.375521 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:01.377630 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.378070 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.378091 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.378092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.378227 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.378490 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.378492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.378492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.378791 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:01.379471 (+ 680us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:01.379549 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 06:36:01.379706 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":115,"threads_started":1}
I20260810 06:36:01.383087 4609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33558536909744c3b2d1bf31554e9446: Generated new TSK 0
I20260810 06:36:01.383236 4609 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:36:01.417632 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33281
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:01.501459 4623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:01.501709 4623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:01.501753 4623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:01.503058 4623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:01.503118 4623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:01.503178 4623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:01.504880 4623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:01.505170 4623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:01.505414 4623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:01.510558 4628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:01.510594 4629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:01.510558 4631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:01.510735 4623 server_base.cc:1034] running on GCE node
I20260810 06:36:01.511112 4623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:01.511377 4623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:01.512547 4623 hybrid_clock.cc:648] HybridClock initialized: now 1786343761512536 us; error 34 us; skew 500 ppm
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.515508 4623 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:36:01.515949 4623 webserver.cc:460] Webserver started at http://127.30.71.1:40869/ using document root <none> and password file <none>
I20260810 06:36:01.516153 4623 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:01.516197 4623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:01.516311 4623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:01.517438 4623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "01089d6f03d441d8b936c5c2a6d65e5d"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "67a657579703be88dcde228fb9641698"
server_key_iv: "e10da57b1c2b98cc59f806b2860f5e6d"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.517766 4623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "01089d6f03d441d8b936c5c2a6d65e5d"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "67a657579703be88dcde228fb9641698"
server_key_iv: "e10da57b1c2b98cc59f806b2860f5e6d"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.519274 4623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:36:01.520205 4638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.520426 4623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:01.520545 4623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "01089d6f03d441d8b936c5c2a6d65e5d"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "67a657579703be88dcde228fb9641698"
server_key_iv: "e10da57b1c2b98cc59f806b2860f5e6d"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.520630 4623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:01.530452 4623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:01.532850 4623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:01.533030 4623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:01.533319 4623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:01.533660 4623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:01.533720 4623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.533769 4623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:01.533793 4623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.540436 4623 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38419
I20260810 06:36:01.540520 4760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38419 every 8 connection(s)
I20260810 06:36:01.540881 4623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:01.543529 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4623
I20260810 06:36:01.543643 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:36:01.543859 31004 external_mini_cluster.cc:1428] Setting key 4d8c7d7dbd2994a2f6f408a5934e3cb2
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:01.552507 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.541686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:35229 (local address 127.30.71.62:33281)
0810 06:36:01.541749 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.541752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.541947 (+ 195us) server_negotiation.cc:407] Connection header received
0810 06:36:01.542955 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.542956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.542969 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.543048 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.543949 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.544616 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.546297 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.546551 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.546646 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.549453 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:01.549454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:01.549454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:01.549461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:01.550893 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.551474 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.551475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.551475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.551529 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.551835 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.551836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.551836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.551940 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:01.551991 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:01.552367 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 06:36:01.552438 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:36:01.552882 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.541644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33281 (local address 127.30.71.1:35229)
0810 06:36:01.541814 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:01.541862 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:01.542837 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:01.543129 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:01.543133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:01.543515 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:01.543820 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:01.543828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.544819 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.544821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:01.546180 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:01.546184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.546655 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.546657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:01.546816 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.547452 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:01.547461 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:01.549346 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:01.551015 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:01.551025 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:01.551028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:01.551388 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:01.551604 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:01.551605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:01.551605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:01.551759 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:01.552011 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:01.552019 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:01.552208 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 06:36:01.552349 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 06:36:01.552535 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260810 06:36:01.553505 4761 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33281
I20260810 06:36:01.553624 4761 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:01.553844 4761 heartbeater.cc:511] Master 127.30.71.62:33281 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:36:01.554456 4555 ts_manager.cc:194] Registered new tserver with Master: 01089d6f03d441d8b936c5c2a6d65e5d (127.30.71.1:38419)
I20260810 06:36:01.555191 4555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:35229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:36:01.573715 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33281
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:01.658022 4766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:01.658366 4766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:01.658409 4766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:01.659624 4766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:01.659700 4766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:01.659744 4766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:01.661267 4766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:01.661536 4766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:01.661780 4766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:01.666927 4775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:01.666929 4772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:01.666927 4771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:01.667353 4766 server_base.cc:1034] running on GCE node
I20260810 06:36:01.667569 4766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:01.667771 4766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:01.668913 4766 hybrid_clock.cc:648] HybridClock initialized: now 1786343761668903 us; error 33 us; skew 500 ppm
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.671741 4766 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:36:01.672217 4766 webserver.cc:460] Webserver started at http://127.30.71.2:39381/ using document root <none> and password file <none>
I20260810 06:36:01.672426 4766 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:01.672467 4766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:01.672564 4766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:01.673574 4766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "0f470c6f2ddf4a72bb982c35ef8e2533"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "5da9568aa361bf46bdf1b1948e037684"
server_key_iv: "ef43f81c3bc962865fb6de6eba2c1893"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.673878 4766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "0f470c6f2ddf4a72bb982c35ef8e2533"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "5da9568aa361bf46bdf1b1948e037684"
server_key_iv: "ef43f81c3bc962865fb6de6eba2c1893"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.675271 4766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:36:01.675974 4783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.676146 4766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:01.676261 4766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "0f470c6f2ddf4a72bb982c35ef8e2533"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "5da9568aa361bf46bdf1b1948e037684"
server_key_iv: "ef43f81c3bc962865fb6de6eba2c1893"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.676337 4766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:01.685598 4766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:01.687734 4766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:01.687892 4766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:01.688123 4766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:01.688432 4766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:01.688485 4766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.688527 4766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:01.688555 4766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.694603 4766 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:36581
I20260810 06:36:01.694643 4902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:36581 every 8 connection(s)
I20260810 06:36:01.695001 4766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:36:01.699209 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4766
I20260810 06:36:01.699302 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:36:01.699505 31004 external_mini_cluster.cc:1428] Setting key 77837ca0894b956c97db9bbea4295cae
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:01.705499 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.695817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:58251 (local address 127.30.71.62:33281)
0810 06:36:01.695885 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.695887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.696090 (+ 203us) server_negotiation.cc:407] Connection header received
0810 06:36:01.697117 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.697120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.697134 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.697186 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.697836 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.698338 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.699427 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.699675 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.699751 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.702448 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:01.702449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:01.702449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:01.702458 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:01.703963 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.704457 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.704459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.704459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.704521 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.704812 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.704813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.704813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.704940 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:01.705016 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:01.705314 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 06:36:01.705399 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:36:01.705717 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.695797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33281 (local address 127.30.71.2:58251)
0810 06:36:01.695955 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:01.695998 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:01.696980 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:01.697262 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:01.697266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:01.697540 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:01.697725 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:01.697729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.698436 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.698437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:01.699328 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:01.699332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.699765 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.699766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:01.699944 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.700522 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:01.700534 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:01.702344 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:01.704070 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:01.704079 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:01.704081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:01.704363 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:01.704590 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:01.704591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:01.704591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:01.704751 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:01.705019 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:01.705026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:01.705175 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 06:36:01.705295 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:36:01.705469 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260810 06:36:01.706207 4903 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33281
I20260810 06:36:01.706307 4903 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:01.706509 4903 heartbeater.cc:511] Master 127.30.71.62:33281 requested a full tablet report, sending...
I20260810 06:36:01.707010 4555 ts_manager.cc:194] Registered new tserver with Master: 0f470c6f2ddf4a72bb982c35ef8e2533 (127.30.71.2:36581)
I20260810 06:36:01.707408 4555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:58251
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:36:01.726342 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33281
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:43321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:01.810472 4908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:01.810757 4908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:01.810796 4908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:01.812019 4908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:01.812093 4908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:01.812134 4908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:01.813726 4908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:01.814030 4908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:01.814277 4908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:01.819324 4913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:01.819468 4916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:01.819468 4914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:01.819782 4908 server_base.cc:1034] running on GCE node
I20260810 06:36:01.820021 4908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:01.820225 4908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:01.821367 4908 hybrid_clock.cc:648] HybridClock initialized: now 1786343761821355 us; error 30 us; skew 500 ppm
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.824033 4908 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:36:01.824491 4908 webserver.cc:460] Webserver started at http://127.30.71.3:44291/ using document root <none> and password file <none>
I20260810 06:36:01.824680 4908 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:01.824720 4908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:01.824811 4908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:01.825803 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "34253a2d18624f2bac0ae0cba7c5c29c"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "9779f398a255f7b3ce1b11f1ac62f647"
server_key_iv: "f764ef331639f36eeaa705969e3c98a8"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.826102 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "34253a2d18624f2bac0ae0cba7c5c29c"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "9779f398a255f7b3ce1b11f1ac62f647"
server_key_iv: "f764ef331639f36eeaa705969e3c98a8"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.827512 4908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 06:36:01.828236 4923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.828460 4908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:01.828552 4908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "34253a2d18624f2bac0ae0cba7c5c29c"
format_stamp: "Formatted at 2026-08-10 06:36:01 on dist-test-slave-w206"
server_key: "9779f398a255f7b3ce1b11f1ac62f647"
server_key_iv: "f764ef331639f36eeaa705969e3c98a8"
server_key_version: "encryptionkey@0"
I20260810 06:36:01.828611 4908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:01.837860 4908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:01.840013 4908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:01.840134 4908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:01.840381 4908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:01.840651 4908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:01.840682 4908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.840704 4908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:01.840718 4908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:01.846769 4908 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:45685
I20260810 06:36:01.846856 5042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:45685 every 8 connection(s)
I20260810 06:36:01.847266 4908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:36:01.851528 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 4908
I20260810 06:36:01.851630 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:36:01.851828 31004 external_mini_cluster.cc:1428] Setting key bd53d9b2887fdd99e4313bdb8648dc6d
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:01.857061 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.848116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:57833 (local address 127.30.71.62:33281)
0810 06:36:01.848190 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.848192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.848405 (+ 213us) server_negotiation.cc:407] Connection header received
0810 06:36:01.849342 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.849344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.849356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.849416 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.850057 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.850592 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.851886 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.852177 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.852256 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.854396 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:01.854397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:01.854398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:01.854404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:01.855859 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.856216 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.856217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.856217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.856271 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.856527 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.856528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.856528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.856630 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:01.856681 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:01.856934 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 06:36:01.856995 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:36:01.857286 5045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.848068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33281 (local address 127.30.71.3:57833)
0810 06:36:01.848248 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:01.848291 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:01.849235 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:01.849478 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:01.849482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:01.849757 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:01.849964 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:01.849968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.850678 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.850678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:01.851779 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:01.851794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.852273 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.852274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:01.852394 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.852909 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:01.852918 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:01.854296 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:01.855929 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:01.855935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:01.855937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:01.856149 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:01.856363 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:01.856364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:01.856364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:01.856467 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:01.856689 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:01.856695 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:01.856819 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 06:36:01.856904 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:36:01.857026 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260810 06:36:01.857779 5043 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33281
I20260810 06:36:01.857904 5043 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:01.858099 5043 heartbeater.cc:511] Master 127.30.71.62:33281 requested a full tablet report, sending...
I20260810 06:36:01.858493 4555 ts_manager.cc:194] Registered new tserver with Master: 34253a2d18624f2bac0ae0cba7c5c29c (127.30.71.3:45685)
I20260810 06:36:01.858875 4555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:57833
I20260810 06:36:01.863760 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/test-admin.keytab.
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.880612 31004 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 06:36:01.882243 5050 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:01.881453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.30.71.1:38419)
0810 06:36:01.881632 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.881637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.881648 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:01.881674 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.881676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.881690 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.881780 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.882144 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47406: BlockingRecv error: recv got EOF from 127.0.0.1:47406 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
W20260810 06:36:01.882472 31004 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.883817 31004 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.884932 31004 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:36:01.893111 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.885623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.30.71.1:38419)
0810 06:36:01.885683 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.885685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.885714 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:36:01.885792 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.885792 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.885803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.885854 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.886239 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.887055 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.887785 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.888103 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.888161 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.890337 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:01.890345 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:01.890346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:01.890353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:01.891875 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.892215 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.892218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.892219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.892288 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.892508 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.892510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.892510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.892695 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:01.892831 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:01.892878 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 06:36:01.892946 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 06:36:01.895515 5050 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:01.894687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.30.71.1:38419)
0810 06:36:01.894746 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.894748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.894843 (+ 95us) server_negotiation.cc:407] Connection header received
0810 06:36:01.894886 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.894887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.894897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.894942 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.895460 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47428: BlockingRecv error: recv got EOF from 127.0.0.1:47428 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 06:36:01.895632 31004 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.896677 31004 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.897578 31004 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:36:01.905074 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.898090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47440 (local address 127.30.71.1:38419)
0810 06:36:01.898161 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.898163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.898214 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:36:01.898266 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.898267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.898276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.898323 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.898685 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.899291 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.900032 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.900221 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.900305 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.902340 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:01.902341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:01.902341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:01.902347 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:01.903861 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.904236 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.904237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.904237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.904307 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.904544 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.904545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.904545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.904653 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:01.904737 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:01.904854 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 06:36:01.904920 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 06:36:01.906790 5050 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:01.906132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.30.71.1:38419)
0810 06:36:01.906167 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.906169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.906205 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:36:01.906276 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.906277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.906286 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.906334 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.906709 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47442: BlockingRecv error: recv got EOF from 127.0.0.1:47442 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 06:36:01.906904 31004 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786343734291041-31004-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.908059 31004 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:01.909041 31004 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[4503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:36:01.916090 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:01.909578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.30.71.1:38419)
0810 06:36:01.909636 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:01.909638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:01.909694 (+ 56us) server_negotiation.cc:407] Connection header received
0810 06:36:01.909740 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:01.909740 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:01.909748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:01.909797 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:01.910156 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.910620 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.911384 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:01.911563 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:01.911620 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:01.913627 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:01.913628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:01.913629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:01.913635 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:01.915132 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.915428 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.915429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.915429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.915478 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:01.915636 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:01.915637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:01.915637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:01.915723 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:01.915803 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:01.915889 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 06:36:01.915945 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:36:01.916850 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4623
I20260810 06:36:01.923280 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4766
I20260810 06:36:01.930859 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4908
I20260810 06:36:01.936503 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 4518
2026-08-10T06:36:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1958 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[5066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[5066](info): set up 2 sockets
Aug 10 06:36:01 dist-test-slave-w206 krb5kdc[5066](info): commencing operation
krb5kdc: starting...
W20260810 06:36:03.050097 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:36:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab.
Entry for principal oryx/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab.
I20260810 06:36:03.174448 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:35649
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab
--principal=oryx/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:35649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:03.256995 5081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:03.257277 5081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:03.257318 5081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:03.258545 5081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:03.258622 5081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:03.258646 5081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:03.258668 5081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:03.258698 5081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:03.260290 5081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:35649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab
--principal=oryx/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:35649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:03.260591 5081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:03.260834 5081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:03.266007 5086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.266064 5081 server_base.cc:1034] running on GCE node
W20260810 06:36:03.266022 5089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:03.266031 5087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.266542 5081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:03.266821 5081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:03.268008 5081 hybrid_clock.cc:648] HybridClock initialized: now 1786343763267991 us; error 50 us; skew 500 ppm
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:03.270783 5081 init.cc:377] Logged in from keytab as oryx/127.30.71.62@KRBTEST.COM (short username oryx)
I20260810 06:36:03.271364 5081 webserver.cc:460] Webserver started at http://127.30.71.62:38307/ using document root <none> and password file <none>
I20260810 06:36:03.271606 5081 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:03.271673 5081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:03.271829 5081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:03.272955 5081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "c6c80a34619c4d6a8f4efaa8738de4bb"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "30434cd5e567b67525bd39042afb0d48"
server_key_iv: "28ebbf85a13a8776c745ca1cae968e33"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.273344 5081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "c6c80a34619c4d6a8f4efaa8738de4bb"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "30434cd5e567b67525bd39042afb0d48"
server_key_iv: "28ebbf85a13a8776c745ca1cae968e33"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.274775 5081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:36:03.275651 5097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.275837 5081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:03.275943 5081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "c6c80a34619c4d6a8f4efaa8738de4bb"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "30434cd5e567b67525bd39042afb0d48"
server_key_iv: "28ebbf85a13a8776c745ca1cae968e33"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.276028 5081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:03.285697 5081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:03.287966 5081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:03.288141 5081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:03.292425 5081 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:35649
I20260810 06:36:03.292464 5155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:35649 every 8 connection(s)
I20260810 06:36:03.292838 5081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:03.293840 5157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:03.296315 5157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: Bootstrap starting.
I20260810 06:36:03.296943 5157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:03.297251 5157 log.cc:824] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:03.298110 5157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: No bootstrap required, opened a new log
I20260810 06:36:03.299969 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 5081
I20260810 06:36:03.299929 5157 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } }
I20260810 06:36:03.300096 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:36:03.300069 5157 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:03.300132 5157 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c80a34619c4d6a8f4efaa8738de4bb, State: Initialized, Role: FOLLOWER
I20260810 06:36:03.300294 5157 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } }
I20260810 06:36:03.300333 31004 external_mini_cluster.cc:1428] Setting key 1a6966ffcf4d9c5f0f97132e00d12762
I20260810 06:36:03.300359 5157 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:03.300408 5157 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:03.300459 5157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:03.301268 5157 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } }
I20260810 06:36:03.301365 5157 leader_election.cc:302] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6c80a34619c4d6a8f4efaa8738de4bb; no voters:
I20260810 06:36:03.301590 5157 leader_election.cc:288] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:36:03.301618 5162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:03.301728 5162 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 1 LEADER]: Becoming Leader. State: Replica: c6c80a34619c4d6a8f4efaa8738de4bb, State: Running, Role: LEADER
I20260810 06:36:03.301813 5162 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } }
I20260810 06:36:03.301920 5157 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:03.302253 5164 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } }
I20260810 06:36:03.302346 5164 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:03.302517 5163 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6c80a34619c4d6a8f4efaa8738de4bb. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } }
I20260810 06:36:03.302578 5163 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:03.302889 5168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:03.303509 5168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:03.305459 5168 catalog_manager.cc:1347] Generated new cluster ID: 004cb194534c42039442ee686847c45b
I20260810 06:36:03.305519 5168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:03.313184 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:03.300942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56402 (local address 127.30.71.62:35649)
0810 06:36:03.301548 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:03.301557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:03.301584 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:36:03.301720 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:03.301727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:03.301802 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:03.302768 (+ 966us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:03.302886 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.304412 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.305400 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.305984 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.306087 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:03.309269 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:03.309274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:03.309275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:03.309287 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:03.311381 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.311832 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.311836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.311836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.311961 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.312198 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.312199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.312199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.312532 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:03.312690 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:03.312820 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 06:36:03.312962 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":512,"threads_started":1}
I20260810 06:36:03.326150 5168 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:36:03.326928 5168 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:03.330907 5168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: Generated new TSK 0
I20260810 06:36:03.331141 5168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab.
Entry for principal oryx/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab.
I20260810 06:36:03.367969 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35649
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab
--principal=oryx/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:03.452240 5185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:03.452503 5185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:03.452544 5185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:03.453706 5185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:03.453773 5185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:03.453816 5185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:03.455305 5185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab
--principal=oryx/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:03.455574 5185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:03.455807 5185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:03.460842 5190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:03.460851 5193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:03.460942 5191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.461206 5185 server_base.cc:1034] running on GCE node
I20260810 06:36:03.461426 5185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:03.461704 5185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:03.462859 5185 hybrid_clock.cc:648] HybridClock initialized: now 1786343763462839 us; error 51 us; skew 500 ppm
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:03.465435 5185 init.cc:377] Logged in from keytab as oryx/127.30.71.1@KRBTEST.COM (short username oryx)
I20260810 06:36:03.465894 5185 webserver.cc:460] Webserver started at http://127.30.71.1:35019/ using document root <none> and password file <none>
I20260810 06:36:03.466080 5185 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:03.466120 5185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:03.466208 5185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:03.467284 5185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "c5813f7c05d041ce8d3ab0673b7db1d0"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "4c1941aeb17c6690ab6012594a43e9e4"
server_key_iv: "c3c94f73eb1cc0561bed44b85d5f0f0b"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.467597 5185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "c5813f7c05d041ce8d3ab0673b7db1d0"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "4c1941aeb17c6690ab6012594a43e9e4"
server_key_iv: "c3c94f73eb1cc0561bed44b85d5f0f0b"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.468865 5185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:03.469630 5200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.469847 5185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:03.469923 5185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "c5813f7c05d041ce8d3ab0673b7db1d0"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "4c1941aeb17c6690ab6012594a43e9e4"
server_key_iv: "c3c94f73eb1cc0561bed44b85d5f0f0b"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.469977 5185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:03.492331 5185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:03.494385 5185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:03.494501 5185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:03.494701 5185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:03.495108 5185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:03.495147 5185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.495172 5185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:03.495214 5185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.501277 5185 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:41201
I20260810 06:36:03.501323 5320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:41201 every 8 connection(s)
I20260810 06:36:03.501677 5185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:03.504096 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 5185
I20260810 06:36:03.504206 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:36:03.504426 31004 external_mini_cluster.cc:1428] Setting key 66336b849b564cba814a38736069c3ce
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:03.513137 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:03.502410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:40399 (local address 127.30.71.62:35649)
0810 06:36:03.502521 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:03.502524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:03.502750 (+ 226us) server_negotiation.cc:407] Connection header received
0810 06:36:03.503344 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:03.503345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:03.503357 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:03.503413 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:03.504763 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.505349 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.506897 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.507257 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.507337 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:03.509952 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:03.509954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:03.509954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:03.509962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:03.511714 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.512115 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.512116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.512116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.512183 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.512424 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.512426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.512426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.512583 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:03.512659 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:03.512913 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:36:03.513006 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:36:03.513154 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:03.502399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:40399)
0810 06:36:03.502610 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:03.502665 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:03.503252 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:03.503476 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:03.503480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:03.504330 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:03.504644 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:03.504650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.505448 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.505449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:03.506791 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:03.506795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.507342 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.507343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:03.507461 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:03.508032 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:03.508040 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:03.509847 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:03.511800 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:03.511807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:03.511809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:03.512034 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:03.512256 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:03.512256 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:03.512256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:03.512355 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:03.512651 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:03.512655 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:03.512775 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 06:36:03.512855 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:36:03.512984 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
I20260810 06:36:03.513724 5321 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35649
I20260810 06:36:03.513842 5321 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:36:03.514051 5321 heartbeater.cc:511] Master 127.30.71.62:35649 requested a full tablet report, sending...
I20260810 06:36:03.514631 5115 ts_manager.cc:194] Registered new tserver with Master: c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1:41201)
I20260810 06:36:03.515349 5115 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:40399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab.
Entry for principal oryx/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab.
I20260810 06:36:03.533983 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35649
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab
--principal=oryx/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:03.617558 5326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:03.617786 5326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:03.617810 5326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:03.618927 5326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:03.618965 5326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:03.619077 5326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:03.620560 5326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab
--principal=oryx/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:03.620766 5326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:03.620950 5326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:03.626036 5335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:03.626056 5333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:03.626034 5332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.626411 5326 server_base.cc:1034] running on GCE node
I20260810 06:36:03.626626 5326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:03.626879 5326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:03.628046 5326 hybrid_clock.cc:648] HybridClock initialized: now 1786343763628046 us; error 47 us; skew 500 ppm
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:03.630710 5326 init.cc:377] Logged in from keytab as oryx/127.30.71.2@KRBTEST.COM (short username oryx)
I20260810 06:36:03.631225 5326 webserver.cc:460] Webserver started at http://127.30.71.2:34105/ using document root <none> and password file <none>
I20260810 06:36:03.631654 5326 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:03.631719 5326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:03.631819 5326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:03.632853 5326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "36f0ddc883dc4e4ea46f3442414adbd4"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "14786148295462eccc84b10d7a4c758c"
server_key_iv: "b4fea4935f03e5429c19987ccbc33266"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.633147 5326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "36f0ddc883dc4e4ea46f3442414adbd4"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "14786148295462eccc84b10d7a4c758c"
server_key_iv: "b4fea4935f03e5429c19987ccbc33266"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.634582 5326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:03.635412 5342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.635653 5326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:36:03.635780 5326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "36f0ddc883dc4e4ea46f3442414adbd4"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "14786148295462eccc84b10d7a4c758c"
server_key_iv: "b4fea4935f03e5429c19987ccbc33266"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.635862 5326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:03.650298 5326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:03.652292 5326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:03.652474 5326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:03.652729 5326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:03.653066 5326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:03.653124 5326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.653160 5326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:03.653209 5326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.659531 5326 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:33097
I20260810 06:36:03.659576 5462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:33097 every 8 connection(s)
I20260810 06:36:03.659929 5326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:03.669147 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:03.660668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:58197 (local address 127.30.71.62:35649)
0810 06:36:03.660740 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:03.660743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:03.660957 (+ 214us) server_negotiation.cc:407] Connection header received
0810 06:36:03.661458 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:03.661460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:03.661472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:03.661580 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:03.662632 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.663178 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.664190 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.664380 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.664439 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:03.666538 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:03.666540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:03.666540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:03.666547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:03.668009 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.668360 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.668360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.668361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.668415 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.668626 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.668626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.668626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.668729 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:03.668780 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:03.669016 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 06:36:03.669080 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:36:03.669309 5465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:03.660649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:58197)
0810 06:36:03.660824 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:03.660867 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:03.661348 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:03.661656 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:03.661660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:03.662249 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:03.662529 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:03.662535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.663267 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.663268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:03.664103 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:03.664106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.664457 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.664458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:03.664561 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:03.665042 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:03.665050 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:03.666453 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:03.668081 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:03.668086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:03.668088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:03.668294 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:03.668474 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:03.668474 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:03.668474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:03.668570 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:03.668794 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:03.668798 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:03.668907 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 06:36:03.668988 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:36:03.669104 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260810 06:36:03.669664 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 5326
I20260810 06:36:03.669782 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:36:03.669910 5463 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35649
I20260810 06:36:03.670017 5463 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:03.670025 31004 external_mini_cluster.cc:1428] Setting key 3e524b62037e48c6e6ae9b2750665fa6
I20260810 06:36:03.670221 5463 heartbeater.cc:511] Master 127.30.71.62:35649 requested a full tablet report, sending...
I20260810 06:36:03.670620 5114 ts_manager.cc:194] Registered new tserver with Master: 36f0ddc883dc4e4ea46f3442414adbd4 (127.30.71.2:33097)
I20260810 06:36:03.671066 5114 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:58197
WARNING: no policy specified for oryx/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab.
Entry for principal oryx/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab.
I20260810 06:36:03.698467 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35649
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab
--principal=oryx/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:03.782780 5468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:03.783051 5468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:03.783116 5468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:03.784400 5468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:03.784446 5468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:03.784497 5468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:03.785995 5468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab
--principal=oryx/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:03.786250 5468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:03.786491 5468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:03.791718 5476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:03.791728 5473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.791743 5468 server_base.cc:1034] running on GCE node
W20260810 06:36:03.791746 5474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.792152 5468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:03.792400 5468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:03.793604 5468 hybrid_clock.cc:648] HybridClock initialized: now 1786343763793573 us; error 46 us; skew 500 ppm
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:03.796562 5468 init.cc:377] Logged in from keytab as oryx/127.30.71.3@KRBTEST.COM (short username oryx)
I20260810 06:36:03.796967 5468 webserver.cc:460] Webserver started at http://127.30.71.3:46545/ using document root <none> and password file <none>
I20260810 06:36:03.797166 5468 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:03.797209 5468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:03.797310 5468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:03.798399 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "d7677fde3633408daf7b0484fd7e5744"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "e5a439efba093ea7aa506928068b3e91"
server_key_iv: "12e5df0b965251b73b48fb54e660c204"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.798725 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "d7677fde3633408daf7b0484fd7e5744"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "e5a439efba093ea7aa506928068b3e91"
server_key_iv: "12e5df0b965251b73b48fb54e660c204"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.800127 5468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:36:03.800900 5483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.801092 5468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:36:03.801189 5468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "d7677fde3633408daf7b0484fd7e5744"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "e5a439efba093ea7aa506928068b3e91"
server_key_iv: "12e5df0b965251b73b48fb54e660c204"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.801303 5468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:03.821842 5468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:03.824661 5468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:03.824795 5468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:03.824973 5468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:03.825312 5468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:03.825356 5468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.825412 5468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:03.825438 5468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.831411 5468 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:40743
I20260810 06:36:03.831521 5603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:40743 every 8 connection(s)
I20260810 06:36:03.831861 5468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:36:03.834295 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 5468
I20260810 06:36:03.834429 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:36:03.834605 31004 external_mini_cluster.cc:1428] Setting key cf8e13c59023148d807a43022ca114bb
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:03.842015 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:03.832633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:49795 (local address 127.30.71.62:35649)
0810 06:36:03.832696 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:03.832698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:03.832911 (+ 213us) server_negotiation.cc:407] Connection header received
0810 06:36:03.833470 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:03.833471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:03.833485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:03.833537 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:03.834579 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.835350 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.836434 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.836647 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.836733 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:03.839162 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:03.839163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:03.839163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:03.839170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:03.840696 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.841127 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.841128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.841128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.841183 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:03.841433 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:03.841434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:03.841434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:03.841539 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:03.841591 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:03.841882 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 06:36:03.841948 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:36:03.842293 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:03.832610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:49795)
0810 06:36:03.832785 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:03.832821 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:03.833348 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:03.833607 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:03.833611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:03.834243 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:03.834481 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:03.834488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.835448 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.835449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:03.836332 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:03.836334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:03.836746 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:03.836747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:03.836890 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:03.837476 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:03.837485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:03.839075 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:03.840811 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:03.840818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:03.840820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:03.841055 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:03.841245 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:03.841245 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:03.841246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:03.841375 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:03.841622 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:03.841628 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:03.841759 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 06:36:03.841855 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:36:03.841978 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260810 06:36:03.842842 5604 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35649
I20260810 06:36:03.842957 5604 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:03.843197 5604 heartbeater.cc:511] Master 127.30.71.62:35649 requested a full tablet report, sending...
I20260810 06:36:03.843581 5114 ts_manager.cc:194] Registered new tserver with Master: d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743)
I20260810 06:36:03.843954 5114 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:49795
I20260810 06:36:03.846444 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.71.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.61.keytab.
Entry for principal oryx/127.30.71.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.61.keytab.
I20260810 06:36:03.874162 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:35069
--webserver_interface=127.30.71.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.61.keytab
--principal=oryx/127.30.71.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:35649,127.30.71.61:35069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:03.957604 5609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:03.957839 5609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:03.957861 5609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:03.959093 5609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:03.959138 5609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:03.959167 5609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:03.959230 5609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:03.959251 5609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:03.960770 5609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:35649,127.30.71.61:35069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.61.keytab
--principal=oryx/127.30.71.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:35069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.71.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
W20260810 06:36:03.960922 5609 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.71.62:35649,127.30.71.61:35069'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 06:36:03.961076 5609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:03.961309 5609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:03.967197 5614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:03.967197 5615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.967252 5609 server_base.cc:1034] running on GCE node
W20260810 06:36:03.967391 5617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:03.967648 5609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:03.967955 5609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:03.969126 5609 hybrid_clock.cc:648] HybridClock initialized: now 1786343763969100 us; error 37 us; skew 500 ppm
Aug 10 06:36:03 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:03.971894 5609 init.cc:377] Logged in from keytab as oryx/127.30.71.61@KRBTEST.COM (short username oryx)
I20260810 06:36:03.972303 5609 webserver.cc:460] Webserver started at http://127.30.71.61:39067/ using document root <none> and password file <none>
I20260810 06:36:03.972571 5609 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:03.972620 5609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:03.972713 5609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:03.973839 5609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/instance:
uuid: "a0f32c2e86ec416e82d5fb893254be3b"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "f67164e9b651188b24fa68ab1058ba54"
server_key_iv: "b3bb710f8f47117a0241ff7127ad868b"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.974144 5609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal/instance:
uuid: "a0f32c2e86ec416e82d5fb893254be3b"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "f67164e9b651188b24fa68ab1058ba54"
server_key_iv: "b3bb710f8f47117a0241ff7127ad868b"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.975540 5609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 06:36:03.976222 5624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:03.976430 5609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:03.976521 5609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
uuid: "a0f32c2e86ec416e82d5fb893254be3b"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "f67164e9b651188b24fa68ab1058ba54"
server_key_iv: "b3bb710f8f47117a0241ff7127ad868b"
server_key_version: "encryptionkey@0"
I20260810 06:36:03.976577 5609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:03.992375 5609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:03.994831 5609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:03.994963 5609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:03.999171 5609 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.61:35069
I20260810 06:36:03.999282 5681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.61:35069 every 8 connection(s)
I20260810 06:36:03.999609 5609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
I20260810 06:36:04.000669 5682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:04.002466 5682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.61@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:04.009603 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 5609
I20260810 06:36:04.013250 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.003305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56416 (local address 127.30.71.62:35649)
0810 06:36:04.003378 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.003380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.003607 (+ 227us) server_negotiation.cc:407] Connection header received
0810 06:36:04.004141 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.004142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.004155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.004300 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.004987 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.005514 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.006578 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.006792 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.006883 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.009197 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.009199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.009199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.009207 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.011353 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.011893 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.011894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.011894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.011952 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.012399 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.012401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.012401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.012610 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.012680 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.013057 (+ 377us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.013150 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:36:04.013396 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.003279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:56416)
0810 06:36:04.003480 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.003517 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.004025 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.004409 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.004413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.004685 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.004885 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.004888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.005624 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.005625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.006455 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.006457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.006893 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.006894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.007113 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.007606 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.007614 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.009094 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.011445 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.011451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.011453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.011801 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.012024 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.012025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.012025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.012279 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.012694 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.012699 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.012868 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.013002 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.013168 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260810 06:36:04.014115 5682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.71.61@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.61@KRBTEST.COM for oryx/127.30.71.61@KRBTEST.COM
I20260810 06:36:04.021195 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.010307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.30.71.61:35069)
0810 06:36:04.010557 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.010590 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.010753 (+ 163us) server_negotiation.cc:407] Connection header received
0810 06:36:04.010801 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.010803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.010817 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.010906 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.011710 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.012864 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.014134 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.014609 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.014730 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.017759 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.017768 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.017768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.017775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.019786 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.020190 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.020195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.020195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.020271 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.020542 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.020543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.020544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.020756 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.020943 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.021006 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.021087 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":122,"threads_started":1}
I20260810 06:36:04.023047 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.014440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.0.0.1:36510)
0810 06:36:04.014682 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.014695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.014733 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.014903 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.014905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.015104 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.015211 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.015214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.016226 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.016226 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.016765 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.016767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.017318 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.017319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.017414 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.017873 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.017878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.019407 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.022032 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.022033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.022033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.022192 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.022398 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.022399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.022399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.022453 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.022706 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.022707 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.022745 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.022812 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.022908 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260810 06:36:04.023133 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.014517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.30.71.61:35069)
0810 06:36:04.014723 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.014726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.014734 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:04.014798 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.014799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.014810 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.014887 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.015295 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.016105 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.016854 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.017217 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.017305 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.019497 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.019499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.019499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.019506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.021954 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.022262 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.022263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.022263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.022318 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.022524 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.022524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.022525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.022637 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.022714 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.022838 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.022959 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":183,"thread_start_us":126,"threads_started":1}
I20260810 06:36:04.024516 5682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Bootstrap starting.
I20260810 06:36:04.025168 5682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:04.025465 5682 log.cc:824] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:04.026219 5682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: No bootstrap required, opened a new log
I20260810 06:36:04.027742 5682 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } }
I20260810 06:36:04.027868 5682 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:04.027900 5682 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f32c2e86ec416e82d5fb893254be3b, State: Initialized, Role: FOLLOWER
I20260810 06:36:04.028065 5682 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } }
I20260810 06:36:04.028386 5682 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:04.028384 5689 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } } }
I20260810 06:36:04.028594 5689 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:36:04.030149 5702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:36:04.030231 5702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:36:04.032116 5609 master_runner.cc:428] Detected that this master a0f32c2e86ec416e82d5fb893254be3b is joining an existing cluster
I20260810 06:36:04.032203 5609 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.71.61:35069
I20260810 06:36:04.040316 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.033493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36512 (local address 127.30.71.61:35069)
0810 06:36:04.033533 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.033535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.033652 (+ 117us) server_negotiation.cc:407] Connection header received
0810 06:36:04.033725 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.033725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.033733 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.033776 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.034117 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.034603 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.035433 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.035658 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.035743 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.037015 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.037017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.037017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.037024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.038791 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.039408 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.039409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.039410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.039477 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.039698 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.039699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.039699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.039822 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.039912 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.040069 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.040234 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":24}
I20260810 06:36:04.040555 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.033299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56426 (local address 127.30.71.62:35649)
0810 06:36:04.033491 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.033493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.033503 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:04.033564 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.033565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.033577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.033640 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.034144 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.034914 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.036042 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.036327 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.036390 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.037477 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.037478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.037479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.037486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.039229 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.039668 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.039669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.039669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.039731 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.039973 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.039974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.039974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.040104 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.040149 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.040370 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.040453 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260810 06:36:04.040509 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.033268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.0.0.1:36512)
0810 06:36:04.033601 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.033609 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.033671 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.033778 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.033779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.033982 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.034037 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.034039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.034684 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.034685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.035320 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.035322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.035744 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.035744 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.035809 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.036475 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.036479 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.036908 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.038891 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.038891 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.038892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.039149 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.039566 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.039567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.039567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.039627 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.039905 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.039905 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.039960 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.040045 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.040165 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":98,"threads_started":1}
I20260810 06:36:04.040908 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.033245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:56426)
0810 06:36:04.033429 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.033442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.033498 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.033658 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.033659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.033945 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.034035 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.034038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.035044 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.035044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.035925 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.035927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.036406 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.036407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.036490 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.036932 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.036937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.037369 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.039351 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.039352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.039352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.039576 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.039806 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.039807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.039807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.039881 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.040210 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.040210 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.040256 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.040688 (+ 432us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.040791 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":125,"threads_started":1}
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.62@KRBTEST.COM for oryx/127.30.71.61@KRBTEST.COM
I20260810 06:36:04.050588 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.043603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.30.71.61:35069)
0810 06:36:04.043639 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.043641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.043828 (+ 187us) server_negotiation.cc:407] Connection header received
0810 06:36:04.043924 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.043925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.043936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.043992 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.044438 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.044906 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.045685 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.045892 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.045967 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.048012 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.048014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.048014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.048022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.049327 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.049737 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.049739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.049739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.049802 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.050020 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.050021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.050021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.050136 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.050210 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.050398 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.050484 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 06:36:04.050621 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.043548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.0.0.1:36516)
0810 06:36:04.043695 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.043762 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.043854 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.043998 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.044001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.044284 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.044334 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.044336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.044986 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.044987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.045580 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.045582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.045967 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.045968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.046048 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.046528 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.046533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.047921 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.049420 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.049428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.049430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.049644 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.049872 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.049873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.049873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.049943 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.050204 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.050212 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.050268 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.050371 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.050503 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260810 06:36:04.051476 5114 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.71.61:35069: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b"
member_type: NON_VOTER
last_known_addr {
host: "127.30.71.61"
port: 35069
}
attrs {
promote: true
}
}
dest_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb"
cas_config_opid_index: -1
I20260810 06:36:04.051864 5114 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } }
I20260810 06:36:04.052376 5712 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } }
I20260810 06:36:04.052739 5654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:04.053561 5712 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.71.61:35069
I20260810 06:36:04.053746 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } }
I20260810 06:36:04.053826 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.053819 5654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Refusing update from remote peer c6c80a34619c4d6a8f4efaa8738de4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 06:36:04.053988 5609 master.cc:537] Master@127.30.71.61:35069 shutting down...
I20260810 06:36:04.054418 5713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:36:04.055217 5100 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 06:36:04.055619 5100 consensus_peers.cc:487] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb -> Peer a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61:35069): Couldn't send request to peer a0f32c2e86ec416e82d5fb893254be3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:36:04.057190 5609 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:04.057287 5609 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:04.057341 5609 tablet_replica.cc:330] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: stopping tablet replica
I20260810 06:36:04.070875 5609 master.cc:552] Master@127.30.71.61:35069 shutdown complete.
I20260810 06:36:04.070937 5609 master_runner.cc:305] Clearing existing system tablet
I20260810 06:36:04.071138 5609 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.071182 5609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.071844 5609 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:04.072417 5718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.072590 5609 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.072650 5609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:36:04.072674 5609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
uuid: "a0f32c2e86ec416e82d5fb893254be3b"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "f67164e9b651188b24fa68ab1058ba54"
server_key_iv: "b3bb710f8f47117a0241ff7127ad868b"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.073428 5609 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:04.074908 5609 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:36:04.075074 5609 log.cc:1196] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 06:36:04.075271 5609 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Deleting consensus metadata
I20260810 06:36:04.075435 5609 master_runner.cc:315] Copying system tablet from 127.30.71.62:35649
I20260810 06:36:04.076107 5609 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: tablet copy: Beginning tablet copy session from remote peer at address 127.30.71.62:35649
I20260810 06:36:04.082877 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.076527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:56430)
0810 06:36:04.076636 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.076647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.076718 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.076869 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.076870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.077040 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.077103 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.077105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.078052 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.078053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.078604 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.078606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.079142 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.079142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.079203 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.079690 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.079695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.080140 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.081824 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.081825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.081825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.081997 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.082258 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.082259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.082259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.082342 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.082620 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.082621 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.082660 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.082712 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.082795 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260810 06:36:04.082913 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.076591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56430 (local address 127.30.71.62:35649)
0810 06:36:04.076644 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.076646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.076699 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:36:04.076789 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.076790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.076802 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.076866 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.077185 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.077964 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.078719 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.079067 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.079141 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.080231 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.080233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.080233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.080241 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.081746 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.082110 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.082111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.082112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.082186 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.082415 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.082416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.082416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.082556 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.082627 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.082739 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.082818 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:36:04.083329 5138 tablet_copy_service.cc:140] P c6c80a34619c4d6a8f4efaa8738de4bb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a0f32c2e86ec416e82d5fb893254be3b ({username='oryx', principal='oryx/127.30.71.61@KRBTEST.COM'} at 127.0.0.1:56430)
I20260810 06:36:04.083515 5138 tablet_copy_service.cc:161] P c6c80a34619c4d6a8f4efaa8738de4bb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a0f32c2e86ec416e82d5fb893254be3b at {username='oryx', principal='oryx/127.30.71.61@KRBTEST.COM'} at 127.0.0.1:56430: session id = a0f32c2e86ec416e82d5fb893254be3b-00000000000000000000000000000000
I20260810 06:36:04.084239 5138 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:36:04.084856 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:36:04.085951 5609 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: tablet copy: Starting download of 0 data blocks...
I20260810 06:36:04.086107 5609 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: tablet copy: Starting download of 1 WAL segments...
I20260810 06:36:04.087117 5609 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:36:04.087949 5138 tablet_copy_service.cc:342] P c6c80a34619c4d6a8f4efaa8738de4bb: Request end of tablet copy session a0f32c2e86ec416e82d5fb893254be3b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.71.61@KRBTEST.COM'} at 127.0.0.1:56430
I20260810 06:36:04.088007 5138 tablet_copy_service.cc:433] P c6c80a34619c4d6a8f4efaa8738de4bb: ending tablet copy session a0f32c2e86ec416e82d5fb893254be3b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a0f32c2e86ec416e82d5fb893254be3b
I20260810 06:36:04.090760 5609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:04.091969 5729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.092064 5727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.092185 5725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:04.095142 5609 server_base.cc:1034] running on GCE node
I20260810 06:36:04.095252 5609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:04.095427 5609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:04.096566 5609 hybrid_clock.cc:648] HybridClock initialized: now 1786343764096559 us; error 27 us; skew 500 ppm
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:04.098107 5609 init.cc:377] Logged in from keytab as oryx/127.30.71.61@KRBTEST.COM (short username oryx)
I20260810 06:36:04.098436 5609 webserver.cc:460] Webserver started at http://127.30.71.61:33855/ using document root <none> and password file <none>
I20260810 06:36:04.098560 5609 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.098611 5609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.099303 5609 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:36:04.099846 5736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.100045 5609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:36:04.100106 5609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
uuid: "a0f32c2e86ec416e82d5fb893254be3b"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "f67164e9b651188b24fa68ab1058ba54"
server_key_iv: "b3bb710f8f47117a0241ff7127ad868b"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.100168 5609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.109582 5609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:04.111518 5609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:04.114813 5609 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.61:35069
I20260810 06:36:04.115247 5792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.61:35069 every 8 connection(s)
I20260810 06:36:04.115302 5609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
I20260810 06:36:04.116169 5793 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:36:04.116627 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Bootstrap starting.
I20260810 06:36:04.119129 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:36:04.119374 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: Bootstrap complete.
I20260810 06:36:04.119589 5793 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } }
I20260810 06:36:04.119660 5793 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a0f32c2e86ec416e82d5fb893254be3b, State: Initialized, Role: LEARNER
I20260810 06:36:04.119688 5793 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } }
I20260810 06:36:04.119947 5793 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:04.119912 5796 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } }
I20260810 06:36:04.120012 5796 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: LEARNER
I20260810 06:36:04.121491 5809 catalog_manager.cc:1259] Loaded cluster ID: 004cb194534c42039442ee686847c45b
I20260810 06:36:04.121569 5809 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: loading cluster ID for follower catalog manager: success
I20260810 06:36:04.122490 5809 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: acquiring CA information for follower catalog manager: success
I20260810 06:36:04.122826 5809 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:36:04.130005 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.123010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.30.71.61:35069)
0810 06:36:04.123130 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.123132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.123142 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:04.123172 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.123173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.123181 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.123252 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.123941 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.124572 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.125608 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.125804 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.125858 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.127394 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.127395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.127396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.127402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.128850 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.129263 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.129264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.129264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.129316 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.129529 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.129529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.129530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.129628 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.129715 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.129856 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.129921 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":48,"threads_started":1}
WARNING: no policy specified for oryx/127.30.71.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.60.keytab.
Entry for principal oryx/127.30.71.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.60.keytab.
I20260810 06:36:04.159166 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:41963
--webserver_interface=127.30.71.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.60.keytab
--principal=oryx/127.30.71.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:04.249038 5813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:04.249284 5813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:04.249325 5813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:04.250530 5813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:04.250598 5813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:04.250623 5813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:04.250656 5813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:04.250684 5813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:04.252302 5813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.60.keytab
--principal=oryx/127.30.71.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:41963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.71.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:04.252614 5813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:04.252856 5813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:04.257797 5821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.257807 5819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.257819 5818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:04.258179 5813 server_base.cc:1034] running on GCE node
I20260810 06:36:04.258392 5813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:04.258653 5813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:04.259790 5813 hybrid_clock.cc:648] HybridClock initialized: now 1786343764259777 us; error 29 us; skew 500 ppm
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:04.262351 5813 init.cc:377] Logged in from keytab as oryx/127.30.71.60@KRBTEST.COM (short username oryx)
I20260810 06:36:04.262866 5813 webserver.cc:460] Webserver started at http://127.30.71.60:37067/ using document root <none> and password file <none>
I20260810 06:36:04.263168 5813 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.263218 5813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.263337 5813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:04.264361 5813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/instance:
uuid: "f2d7a39f822b432996902930410e53f4"
format_stamp: "Formatted at 2026-08-10 06:36:04 on dist-test-slave-w206"
server_key: "f9288d1c7865ec4d1430dbf0d92a5582"
server_key_iv: "a43e817a4480b12618bc0ab9b10f90a5"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.264667 5813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal/instance:
uuid: "f2d7a39f822b432996902930410e53f4"
format_stamp: "Formatted at 2026-08-10 06:36:04 on dist-test-slave-w206"
server_key: "f9288d1c7865ec4d1430dbf0d92a5582"
server_key_iv: "a43e817a4480b12618bc0ab9b10f90a5"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.266068 5813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:04.266808 5829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.267041 5813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:04.267122 5813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
uuid: "f2d7a39f822b432996902930410e53f4"
format_stamp: "Formatted at 2026-08-10 06:36:04 on dist-test-slave-w206"
server_key: "f9288d1c7865ec4d1430dbf0d92a5582"
server_key_iv: "a43e817a4480b12618bc0ab9b10f90a5"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.267176 5813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.275915 5813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:04.278083 5813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:04.278265 5813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:04.283468 5813 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.60:41963
I20260810 06:36:04.283531 5884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.60:41963 every 8 connection(s)
I20260810 06:36:04.283893 5813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
I20260810 06:36:04.285069 5885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:04.285676 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 5813
I20260810 06:36:04.287904 5885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.71.60@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.60@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:04.297828 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.288918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.30.71.62:35649)
0810 06:36:04.288993 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.288995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.289007 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:36:04.289137 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.289138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.289150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.289202 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.289715 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.290172 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.291126 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.291440 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.291496 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.294608 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.294610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.294610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.294620 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.296509 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.296942 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.296942 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.296943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.297024 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.297214 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.297215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.297215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.297351 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.297419 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.297638 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.297727 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:36:04.297996 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.286384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.30.71.60:41963)
0810 06:36:04.286583 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.286590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.286617 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:36:04.287121 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.287128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.287178 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.287372 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.288133 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.289439 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.290230 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.290701 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.290782 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.293961 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.293969 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.293970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.293977 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.296274 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.296686 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.296689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.296689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.296771 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.296993 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.296994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.296994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.297340 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.297492 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.297624 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.297747 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260810 06:36:04.298005 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.288605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:56446)
0810 06:36:04.288904 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.288957 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.289027 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.289219 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.289222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.289557 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.289630 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.289632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.290307 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.290308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.291023 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.291025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.291626 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.291628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.291720 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.292237 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.292248 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.294506 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.296593 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.296601 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.296604 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.296854 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.297096 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.297097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.297097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.297151 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.297431 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.297438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.297508 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.297603 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.297845 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":71,"thread_start_us":117,"threads_started":1}
I20260810 06:36:04.298606 5885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.60@KRBTEST.COM for oryx/127.30.71.61@KRBTEST.COM
I20260810 06:36:04.305605 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.298871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.0.0.1:36546)
0810 06:36:04.298927 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.298939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.299001 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.299176 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.299177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.299316 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.299364 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.299366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.300040 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.300041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.300589 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.300590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.301057 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.301058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.301138 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.301656 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.301661 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.302950 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.304602 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.304603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.304603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.304775 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.304982 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.304983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.304983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.305068 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.305359 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.305360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.305400 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.305454 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.305535 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 06:36:04.305615 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.298968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36546 (local address 127.30.71.61:35069)
0810 06:36:04.299056 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.299058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.299066 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:04.299092 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.299093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.299104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.299175 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.299447 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.299955 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.300673 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.300982 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.301061 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.303124 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.303125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.303126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.303133 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.304500 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.304850 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.304851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.304852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.304915 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.305143 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.305144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.305144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.305292 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.305354 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.305473 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.305551 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:36:04.306123 5885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.60@KRBTEST.COM for oryx/127.30.71.60@KRBTEST.COM
I20260810 06:36:04.312237 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.306414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.0.0.1:34434)
0810 06:36:04.306444 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.306456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.306493 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.306602 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.306603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.306714 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.306745 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.306746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.307407 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.307408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.307866 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.307868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.308175 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.308175 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.308215 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.308529 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.308533 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.309663 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.311436 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.311437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.311438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.311596 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.311773 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.311774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.311774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.311824 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.312010 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.312010 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.312044 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.312093 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.312169 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260810 06:36:04.312355 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.306445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.30.71.60:41963)
0810 06:36:04.306504 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.306506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.306514 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:04.306532 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.306533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.306541 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.306598 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.306803 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.307326 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.307942 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.308125 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.308179 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.309764 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.309765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.309766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.309772 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.311363 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.311666 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.311667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.311667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.311719 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.311862 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.311863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.311863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.311959 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.312009 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.312110 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.312217 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":40}
I20260810 06:36:04.313810 5885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Bootstrap starting.
I20260810 06:36:04.314705 5885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:04.315140 5885 log.cc:824] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:04.315920 5885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: No bootstrap required, opened a new log
I20260810 06:36:04.317601 5885 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } }
I20260810 06:36:04.317732 5885 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:04.317755 5885 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d7a39f822b432996902930410e53f4, State: Initialized, Role: FOLLOWER
I20260810 06:36:04.317901 5885 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } }
I20260810 06:36:04.318266 5885 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:04.318331 5890 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } } }
I20260810 06:36:04.318454 5890 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:36:04.320662 5903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:36:04.320731 5903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:36:04.323307 5813 master_runner.cc:428] Detected that this master f2d7a39f822b432996902930410e53f4 is joining an existing cluster
I20260810 06:36:04.323349 5813 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.71.60:41963
I20260810 06:36:04.331614 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.324627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34446 (local address 127.30.71.60:41963)
0810 06:36:04.324702 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.324703 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.325089 (+ 386us) server_negotiation.cc:407] Connection header received
0810 06:36:04.325132 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.325132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.325143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.325200 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.325691 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.326351 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.327411 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.327683 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.327720 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.328967 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.328968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.328969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.328976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.330425 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.330770 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.330771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.330772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.330826 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.331046 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.331047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.331047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.331162 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.331238 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.331401 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.331496 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:36:04.331696 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.324400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.0.0.1:34446)
0810 06:36:04.325007 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.325018 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.325079 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.325247 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.325248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.325536 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.325609 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.325611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.326427 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.326427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.327327 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.327329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.327825 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.327826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.327893 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.328454 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.328459 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.328840 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.330525 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.330527 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.330527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.330699 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.330901 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.330902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.330903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.330958 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.331229 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.331230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.331288 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.331377 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.331563 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":105,"thread_start_us":137,"threads_started":1}
I20260810 06:36:04.349763 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 5185
I20260810 06:36:04.360291 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.324813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36552 (local address 127.30.71.61:35069)
0810 06:36:04.324928 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.324930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.325221 (+ 291us) server_negotiation.cc:407] Connection header received
0810 06:36:04.325314 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.325315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.325325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.325358 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.325835 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.326629 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.327855 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.328168 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.328218 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.357078 (+ 28860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.357081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.357081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.357092 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.358823 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.359286 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.359288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.359288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.359410 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.359640 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.359641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.359642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.359789 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.359844 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.360078 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.360177 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 06:36:04.361011 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.324764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.0.0.1:36552)
0810 06:36:04.325168 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.325177 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.325244 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.325430 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.325430 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.325676 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.325740 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.325742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.326733 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.326734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.327744 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.327747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.328299 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.328300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.328363 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.356409 (+ 28046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.356417 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.356921 (+ 504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.358938 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.358939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.358940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.359190 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.359496 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.359497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.359497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.359564 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.359903 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.359904 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.359955 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.360714 (+ 759us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.360875 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":177,"threads_started":1}
I20260810 06:36:04.364224 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.324609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56448 (local address 127.30.71.62:35649)
0810 06:36:04.324660 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.324662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.324672 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:04.324810 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.324811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.324823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.324872 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.325410 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.326203 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.327420 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.327716 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.327756 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.357553 (+ 29797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.357556 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.357556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.357567 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.362287 (+ 4720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.362774 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.362775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.362776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.362868 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.363267 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.363268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.363268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.363428 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.363502 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.364028 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.364127 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:36:04.364377 5911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.324383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:56448)
0810 06:36:04.324595 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.324608 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.324665 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.324890 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.324891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.325170 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.325288 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.325290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.326307 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.326307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.327317 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.327319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.327830 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.327830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.327890 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.357069 (+ 29179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.357074 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.357440 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.362402 (+ 4962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.362403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.362404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.362666 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.363022 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.363023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.363023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.363099 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.363783 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.363784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.363849 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.363958 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.364101 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":5,"thread_start_us":144,"threads_started":1}
I20260810 06:36:04.369625 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:41201
--local_ip_for_outbound_sockets=127.30.71.1
--tserver_master_addrs=127.30.71.62:35649
--webserver_port=35019
--webserver_interface=127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab
--principal=oryx/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.62@KRBTEST.COM for oryx/127.30.71.60@KRBTEST.COM
I20260810 06:36:04.375756 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.366948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.0.0.1:34448)
0810 06:36:04.367061 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.367072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.367111 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.367445 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.367447 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.367625 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.367677 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.367680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.368621 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.368623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.369496 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.369498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.370404 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.370405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.370477 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.370955 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.370961 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.372816 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.374614 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.374615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.374615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.374829 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.375073 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.375074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.375074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.375135 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.375408 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.375409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.375457 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.375529 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.375646 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 06:36:04.376430 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.367182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.30.71.60:41963)
0810 06:36:04.367222 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.367224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.367230 (+ 6us) server_negotiation.cc:407] Connection header received
0810 06:36:04.367331 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.367331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.367342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.367390 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.367761 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.368518 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.369618 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.370277 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.370344 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.372932 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.372933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.372934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.372942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.374515 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.374921 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.374923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.374923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.375001 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.375199 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.375200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.375200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.375326 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.375381 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.376252 (+ 871us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.376336 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:36:04.377134 5114 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.71.60:41963: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f2d7a39f822b432996902930410e53f4"
member_type: NON_VOTER
last_known_addr {
host: "127.30.71.60"
port: 41963
}
attrs {
promote: true
}
}
dest_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb"
cas_config_opid_index: 5
I20260810 06:36:04.377403 5114 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } }
I20260810 06:36:04.377772 5712 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f2d7a39f822b432996902930410e53f4 (127.30.71.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.378281 5858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:04.378875 5712 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.71.60:41963
I20260810 06:36:04.378954 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.379066 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.379315 5813 master.cc:537] Master@127.30.71.60:41963 shutting down...
I20260810 06:36:04.379361 5858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Refusing update from remote peer c6c80a34619c4d6a8f4efaa8738de4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 06:36:04.379669 5713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:36:04.380688 5098 consensus_peers.cc:487] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb -> Peer f2d7a39f822b432996902930410e53f4 (127.30.71.60:41963): Couldn't send request to peer f2d7a39f822b432996902930410e53f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:36:04.384792 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.377871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.0.0.1:36560)
0810 06:36:04.380234 (+ 2363us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.380246 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.380302 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.381030 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.381031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.381033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:04.381079 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.381080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.382079 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.382080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.383392 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.383396 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.384450 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.384453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.384531 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.384560 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.384619 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.384670 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2346}
I20260810 06:36:04.385103 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.377941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.30.71.61:35069)
0810 06:36:04.377991 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.377993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.380296 (+ 2303us) server_negotiation.cc:407] Connection header received
0810 06:36:04.380361 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.380362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.380391 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.380468 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:04.381181 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.382005 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.383510 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.384233 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.384315 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.384408 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.384961 (+ 553us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.385015 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:36:04.385340 5763 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 LEARNER]: Refusing update from remote peer c6c80a34619c4d6a8f4efaa8738de4bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 06:36:04.385563 5713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 06:36:04.386202 5763 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f2d7a39f822b432996902930410e53f4 (127.30.71.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.386611 5813 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:04.386713 5813 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:04.386758 5813 tablet_replica.cc:330] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: stopping tablet replica
I20260810 06:36:04.387612 5796 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6c80a34619c4d6a8f4efaa8738de4bb. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.387735 5796 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: LEARNER
I20260810 06:36:04.387792 5796 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: NON_VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: true } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.387852 5796 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: LEARNER
I20260810 06:36:04.388477 5713 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: attempting to promote NON_VOTER a0f32c2e86ec416e82d5fb893254be3b to VOTER
I20260810 06:36:04.388711 5713 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } }
I20260810 06:36:04.389214 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.389113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.0.0.1:34464)
0810 06:36:04.389151 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.389187 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:41963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 06:36:04.389271 5763 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 LEARNER]: Refusing update from remote peer c6c80a34619c4d6a8f4efaa8738de4bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 06:36:04.389518 5712 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 06:36:04.389818 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.389395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.0.0.1:34476)
0810 06:36:04.389755 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.389786 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:41963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345}
I20260810 06:36:04.391065 5813 master.cc:552] Master@127.30.71.60:41963 shutdown complete.
I20260810 06:36:04.391103 5813 master_runner.cc:305] Clearing existing system tablet
I20260810 06:36:04.391233 5813 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.391273 5813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.391368 5763 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
W20260810 06:36:04.389897 5098 consensus_peers.cc:487] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb -> Peer f2d7a39f822b432996902930410e53f4 (127.30.71.60:41963): Couldn't send request to peer f2d7a39f822b432996902930410e53f4. Status: Network error: Client connection negotiation failed: client connection to 127.30.71.60:41963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:36:04.391134 5712 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.392361 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.392439 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:36:04.393107 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.393183 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.393867 5813 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260810 06:36:04.394423 5920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.393244 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.394522 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.394608 5813 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.394665 5813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:36:04.394693 5813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
uuid: "f2d7a39f822b432996902930410e53f4"
format_stamp: "Formatted at 2026-08-10 06:36:04 on dist-test-slave-w206"
server_key: "f9288d1c7865ec4d1430dbf0d92a5582"
server_key_iv: "a43e817a4480b12618bc0ab9b10f90a5"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.395433 5813 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:04.396900 5813 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:36:04.396976 5813 log.cc:1196] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 06:36:04.397193 5813 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Deleting consensus metadata
I20260810 06:36:04.397323 5813 master_runner.cc:315] Copying system tablet from 127.30.71.62:35649
I20260810 06:36:04.397961 5813 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: tablet copy: Beginning tablet copy session from remote peer at address 127.30.71.62:35649
I20260810 06:36:04.405469 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.398338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.30.71.62:35649)
0810 06:36:04.398382 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.398385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.398476 (+ 91us) server_negotiation.cc:407] Connection header received
0810 06:36:04.398542 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.398543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.398554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.398628 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.399115 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.399920 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.400975 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.401271 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.401350 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.402482 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.402483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.402484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.402491 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.404155 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.404594 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.404595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.404596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.404661 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.404873 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.404874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.404874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.405011 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.405080 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.405253 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.405337 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:36:04.405470 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.398282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:56458)
0810 06:36:04.398407 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.398417 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.398479 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.398653 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.398653 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.398886 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.398942 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.398944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.400011 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.400012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.400878 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.400889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.401360 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.401361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.401424 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.401961 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.401967 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.402385 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.404274 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.404275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.404276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.404508 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.404732 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.404733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.404734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.404803 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.405085 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.405086 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.405136 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.405210 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.405367 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260810 06:36:04.406069 5138 tablet_copy_service.cc:140] P c6c80a34619c4d6a8f4efaa8738de4bb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f2d7a39f822b432996902930410e53f4 ({username='oryx', principal='oryx/127.30.71.60@KRBTEST.COM'} at 127.0.0.1:56458)
I20260810 06:36:04.406140 5138 tablet_copy_service.cc:161] P c6c80a34619c4d6a8f4efaa8738de4bb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f2d7a39f822b432996902930410e53f4 at {username='oryx', principal='oryx/127.30.71.60@KRBTEST.COM'} at 127.0.0.1:56458: session id = f2d7a39f822b432996902930410e53f4-00000000000000000000000000000000
I20260810 06:36:04.406713 5138 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:36:04.407331 5813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:36:04.408679 5813 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: tablet copy: Starting download of 0 data blocks...
I20260810 06:36:04.408833 5813 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: tablet copy: Starting download of 1 WAL segments...
I20260810 06:36:04.409627 5813 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:36:04.410570 5138 tablet_copy_service.cc:342] P c6c80a34619c4d6a8f4efaa8738de4bb: Request end of tablet copy session f2d7a39f822b432996902930410e53f4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.71.60@KRBTEST.COM'} at 127.0.0.1:56458
I20260810 06:36:04.410622 5138 tablet_copy_service.cc:433] P c6c80a34619c4d6a8f4efaa8738de4bb: ending tablet copy session f2d7a39f822b432996902930410e53f4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f2d7a39f822b432996902930410e53f4
I20260810 06:36:04.413275 5813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:04.414522 5927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.414561 5929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.414839 5931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:04.415139 5813 server_base.cc:1034] running on GCE node
I20260810 06:36:04.415228 5813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:04.415377 5813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:04.416512 5813 hybrid_clock.cc:648] HybridClock initialized: now 1786343764416503 us; error 26 us; skew 500 ppm
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:04.418533 5813 init.cc:377] Logged in from keytab as oryx/127.30.71.60@KRBTEST.COM (short username oryx)
I20260810 06:36:04.418861 5813 webserver.cc:460] Webserver started at http://127.30.71.60:41835/ using document root <none> and password file <none>
I20260810 06:36:04.419091 5813 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.419133 5813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.419958 5813 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:04.420593 5940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.420733 5813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:04.420774 5813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
uuid: "f2d7a39f822b432996902930410e53f4"
format_stamp: "Formatted at 2026-08-10 06:36:04 on dist-test-slave-w206"
server_key: "f9288d1c7865ec4d1430dbf0d92a5582"
server_key_iv: "a43e817a4480b12618bc0ab9b10f90a5"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.420835 5813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.440834 5813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:04.442737 5813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:04.445961 5813 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.60:41963
I20260810 06:36:04.446245 5997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.60:41963 every 8 connection(s)
I20260810 06:36:04.446353 5813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
I20260810 06:36:04.447248 5998 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:36:04.447723 5998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Bootstrap starting.
I20260810 06:36:04.450706 5998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:36:04.450968 5998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Bootstrap complete.
I20260810 06:36:04.451205 5998 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } }
I20260810 06:36:04.451297 5998 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f2d7a39f822b432996902930410e53f4, State: Initialized, Role: LEARNER
I20260810 06:36:04.451337 5998 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } }
I20260810 06:36:04.451520 5998 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:04.451531 6001 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.451782 6001 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:36:04.453263 6014 catalog_manager.cc:1259] Loaded cluster ID: 004cb194534c42039442ee686847c45b
I20260810 06:36:04.453321 6014 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: loading cluster ID for follower catalog manager: success
I20260810 06:36:04.454610 6014 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: acquiring CA information for follower catalog manager: success
I20260810 06:36:04.454946 6014 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:36:04.479578 5914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:04.479853 5914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:04.479913 5914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:04.481088 5914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:04.481155 5914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:04.481215 5914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:04.482704 5914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab
--principal=oryx/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:41201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=35019
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:04.482968 5914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:04.483258 5914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:04.488394 6019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.488409 6020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.488396 6022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:04.489465 5914 server_base.cc:1034] running on GCE node
I20260810 06:36:04.489703 5914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:04.490031 5914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:04.491192 5914 hybrid_clock.cc:648] HybridClock initialized: now 1786343764491162 us; error 40 us; skew 500 ppm
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:04.494037 5914 init.cc:377] Logged in from keytab as oryx/127.30.71.1@KRBTEST.COM (short username oryx)
I20260810 06:36:04.494606 5914 webserver.cc:460] Webserver started at http://127.30.71.1:35019/ using document root <none> and password file <none>
I20260810 06:36:04.494889 5914 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.494974 5914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.496537 5914 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:04.497253 6029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.497458 5914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:04.497562 5914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "c5813f7c05d041ce8d3ab0673b7db1d0"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "4c1941aeb17c6690ab6012594a43e9e4"
server_key_iv: "c3c94f73eb1cc0561bed44b85d5f0f0b"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.497952 5914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.511512 5914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:04.514354 5914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:04.514546 5914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:04.514832 5914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:04.515269 5914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:04.515331 5914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.515380 5914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:04.515406 5914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.523981 5914 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:41201
I20260810 06:36:04.524029 6151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:41201 every 8 connection(s)
I20260810 06:36:04.524437 5914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:04.530155 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 5914
I20260810 06:36:04.531172 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 5326
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.61@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.60@KRBTEST.COM
I20260810 06:36:04.541272 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:33097
--local_ip_for_outbound_sockets=127.30.71.2
--tserver_master_addrs=127.30.71.62:35649
--webserver_port=34105
--webserver_interface=127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab
--principal=oryx/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:36:04.542459 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.525434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:38125 (local address 127.30.71.62:35649)
0810 06:36:04.525521 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.525525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.525854 (+ 329us) server_negotiation.cc:407] Connection header received
0810 06:36:04.527021 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.527023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.527037 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.527101 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.527893 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.528448 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.529554 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.529814 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.529896 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.538488 (+ 8592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.538489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.538490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.538500 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.540333 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541282 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.541284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.541284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.541353 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541588 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.541589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.541590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.541766 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.541822 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.542253 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.542354 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:36:04.542466 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.525474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:49815 (local address 127.30.71.61:35069)
0810 06:36:04.525523 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.525525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.525837 (+ 312us) server_negotiation.cc:407] Connection header received
0810 06:36:04.527038 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.527040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.527054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.527091 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.527903 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.528693 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.529781 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.530060 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.530100 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.537505 (+ 7405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.537507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.537508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.537517 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.539507 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.540952 (+ 1445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.540953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.540954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.541022 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541702 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.541703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.541704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.541878 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.541930 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.542223 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.542319 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:36:04.542631 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.525437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:53277 (local address 127.30.71.60:41963)
0810 06:36:04.525635 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.525637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.525972 (+ 335us) server_negotiation.cc:407] Connection header received
0810 06:36:04.527038 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.527040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.527054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.527100 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.527903 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.528893 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.529844 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.530172 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.530262 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.538462 (+ 8200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.538464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.538464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.538475 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.541219 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541644 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.541645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.541645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.541706 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541945 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.541946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.541946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.542077 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.542128 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.542379 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.542472 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":131,"threads_started":1}
I20260810 06:36:04.543081 6160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.525224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.30.71.1:53277)
0810 06:36:04.525912 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.525921 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.526905 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.527184 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.527190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.527597 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.527788 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.527794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.528979 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.528979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.529762 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.529764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.530327 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.530329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.530398 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.530998 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.531004 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.538324 (+ 7320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.541312 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.541314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.541533 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.541793 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.541794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.541863 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.542198 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.542199 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.542253 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.542753 (+ 500us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.542842 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":43,"thread_start_us":603,"threads_started":1}
I20260810 06:36:04.543183 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.525231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:38125)
0810 06:36:04.525694 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.525741 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.526874 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.527189 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.527190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.527573 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.527787 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.527791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.528568 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.528569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.529453 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.529454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.529900 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.529900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.530044 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.530660 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.530668 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.538370 (+ 7702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.540953 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.540953 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.540954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.541178 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.541440 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.541441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.541501 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.541875 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.541883 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.542070 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.542571 (+ 501us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.542797 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"spinlock_wait_cycles":8064,"thread_start_us":415,"threads_started":1}
I20260810 06:36:04.543416 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.525225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.30.71.1:49815)
0810 06:36:04.525715 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.525741 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.526905 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.527194 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.527195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.527596 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.527788 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.527794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.528785 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.528786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.529686 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.529688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.530190 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.530191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.530258 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.530794 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.530799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.537378 (+ 6579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.540471 (+ 3093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.540482 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.540485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.540848 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.541106 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.541107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.541107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.541264 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.541984 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.541984 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.542072 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.543132 (+ 1060us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.543273 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":5,"thread_start_us":388,"threads_started":1}
I20260810 06:36:04.543937 6153 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35649
I20260810 06:36:04.543948 6152 heartbeater.cc:347] Connected to a master server at 127.30.71.60:41963
I20260810 06:36:04.544056 6153 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.544129 6154 heartbeater.cc:347] Connected to a master server at 127.30.71.61:35069
I20260810 06:36:04.544204 6154 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.544268 6153 heartbeater.cc:511] Master 127.30.71.62:35649 requested a full tablet report, sending...
I20260810 06:36:04.544288 6154 heartbeater.cc:511] Master 127.30.71.61:35069 requested a full tablet report, sending...
I20260810 06:36:04.544057 6152 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.544538 6152 heartbeater.cc:511] Master 127.30.71.60:41963 requested a full tablet report, sending...
I20260810 06:36:04.544793 5114 ts_manager.cc:194] Re-registered known tserver with Master: c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1:41201)
I20260810 06:36:04.544950 5960 ts_manager.cc:194] Registered new tserver with Master: c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1:41201)
I20260810 06:36:04.545295 5114 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:38125
I20260810 06:36:04.545611 5753 ts_manager.cc:194] Registered new tserver with Master: c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1:41201)
W20260810 06:36:04.628124 6161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:04.628346 6161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:04.628371 6161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:04.629556 6161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:04.629598 6161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:04.629653 6161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:04.631192 6161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab
--principal=oryx/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:33097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=34105
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:04.631428 6161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:04.631665 6161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:04.636897 6166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.636900 6167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.636868 6169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:04.637208 6161 server_base.cc:1034] running on GCE node
I20260810 06:36:04.637435 6161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:04.637692 6161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:04.638832 6161 hybrid_clock.cc:648] HybridClock initialized: now 1786343764638812 us; error 36 us; skew 500 ppm
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:04.641500 6161 init.cc:377] Logged in from keytab as oryx/127.30.71.2@KRBTEST.COM (short username oryx)
I20260810 06:36:04.641952 6161 webserver.cc:460] Webserver started at http://127.30.71.2:34105/ using document root <none> and password file <none>
I20260810 06:36:04.642189 6161 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.642272 6161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.643719 6161 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:36:04.644340 6176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.644516 6161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.644614 6161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "36f0ddc883dc4e4ea46f3442414adbd4"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "14786148295462eccc84b10d7a4c758c"
server_key_iv: "b4fea4935f03e5429c19987ccbc33266"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.644975 6161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.654636 6161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:04.656874 6161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:04.657089 6161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:04.657342 6161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:04.657663 6161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:04.657721 6161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.657765 6161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:04.657790 6161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.666028 6161 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:33097
I20260810 06:36:04.666071 6294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:33097 every 8 connection(s)
I20260810 06:36:04.666464 6161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:36:04.668125 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 6161
I20260810 06:36:04.668291 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 5468
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.61@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.60@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:04.680454 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:40743
--local_ip_for_outbound_sockets=127.30.71.3
--tserver_master_addrs=127.30.71.62:35649
--webserver_port=46545
--webserver_interface=127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab
--principal=oryx/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:36:04.682471 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.667632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:54829 (local address 127.30.71.61:35069)
0810 06:36:04.667752 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.667753 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.667763 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:04.671036 (+ 3273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.671038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.671054 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.671082 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.672059 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.672836 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.674408 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.674694 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.674740 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.678813 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.678814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.678815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.678823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.680763 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.681295 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.681296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.681297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.681357 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.681657 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.681658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.681658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.681810 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.681859 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.682286 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.682370 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 06:36:04.682691 6299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.667221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.30.71.2:54829)
0810 06:36:04.667411 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.667456 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.670768 (+ 3312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.671226 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.671230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.671654 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.671945 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.671950 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.672980 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.672981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.674313 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.674316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.675149 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.675151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.675321 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.675957 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.675968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.678698 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.680860 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.680868 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.680871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.681205 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.681442 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.681443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.681444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.681586 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.681911 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.681918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.682107 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.682250 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.682435 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"spinlock_wait_cycles":7936,"thread_start_us":99,"threads_started":1}
I20260810 06:36:04.682847 6302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.667220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:50215)
0810 06:36:04.667681 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.667692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.670862 (+ 3170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.674804 (+ 3942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.674805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.675010 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.675080 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.675082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.675992 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.675993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.676836 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.676839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.677238 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.677239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.677303 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.677714 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.677719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.679279 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.681573 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.681574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.681575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.681885 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.682111 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.682112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.682112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.682168 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.682454 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.682455 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.682499 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.682566 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.682707 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":109,"threads_started":1}
I20260810 06:36:04.683270 6297 heartbeater.cc:347] Connected to a master server at 127.30.71.61:35069
I20260810 06:36:04.683313 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.667564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:50215 (local address 127.30.71.62:35649)
0810 06:36:04.667681 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.667683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.667752 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:36:04.674603 (+ 6851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.674605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.674621 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.674687 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.675186 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.675899 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.676934 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.677142 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.677235 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.679383 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.679385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.679385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.679393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.681146 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.681973 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.681975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.681975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.682042 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.682235 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.682236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.682236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.682370 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.682429 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.683111 (+ 682us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.683195 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 06:36:04.683388 6297 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.683595 6297 heartbeater.cc:511] Master 127.30.71.61:35069 requested a full tablet report, sending...
I20260810 06:36:04.683598 6296 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35649
I20260810 06:36:04.683691 6296 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.683759 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.667539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:46897 (local address 127.30.71.60:41963)
0810 06:36:04.667588 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.667590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.667780 (+ 190us) server_negotiation.cc:407] Connection header received
0810 06:36:04.671475 (+ 3695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.671477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.671494 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.671525 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.672061 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.672846 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.675320 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.675637 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.675688 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.678811 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.678812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.678812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.678820 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.681540 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.682824 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.682826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.682826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.682905 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.683175 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.683176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.683177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.683323 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.683371 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.683580 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.683680 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:36:04.683768 6296 heartbeater.cc:511] Master 127.30.71.62:35649 requested a full tablet report, sending...
I20260810 06:36:04.684005 5114 ts_manager.cc:194] Re-registered known tserver with Master: 36f0ddc883dc4e4ea46f3442414adbd4 (127.30.71.2:33097)
I20260810 06:36:04.684069 6301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.667221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.30.71.2:46897)
0810 06:36:04.667712 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.667722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.671375 (+ 3653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.671599 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.671600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.671782 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.671944 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.671950 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.672980 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.672981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.675232 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.675234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.675766 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.675766 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.675829 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.676243 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.676248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.678695 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.681629 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.681630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.681630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.682723 (+ 1093us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.683005 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.683006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.683006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.683071 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.683423 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.683423 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.683465 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.683840 (+ 375us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.683972 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":1291,"thread_start_us":98,"threads_started":1}
I20260810 06:36:04.684096 5753 ts_manager.cc:194] Registered new tserver with Master: 36f0ddc883dc4e4ea46f3442414adbd4 (127.30.71.2:33097)
I20260810 06:36:04.684499 5114 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:50215
I20260810 06:36:04.684648 6295 heartbeater.cc:347] Connected to a master server at 127.30.71.60:41963
I20260810 06:36:04.684701 6295 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.684767 6295 heartbeater.cc:511] Master 127.30.71.60:41963 requested a full tablet report, sending...
I20260810 06:36:04.685039 5960 ts_manager.cc:194] Registered new tserver with Master: 36f0ddc883dc4e4ea46f3442414adbd4 (127.30.71.2:33097)
W20260810 06:36:04.766460 6303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:04.766676 6303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:04.766700 6303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:04.767823 6303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:04.767861 6303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:04.767886 6303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:04.769302 6303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab
--principal=oryx/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:40743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=46545
--tserver_master_addrs=127.30.71.62:35649,127.30.71.61:35069,127.30.71.60:41963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:04.769496 6303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:04.769667 6303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:04.774883 6308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.774905 6309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:04.774889 6311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:04.774961 6303 server_base.cc:1034] running on GCE node
I20260810 06:36:04.775319 6303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:04.775584 6303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:04.776751 6303 hybrid_clock.cc:648] HybridClock initialized: now 1786343764776737 us; error 27 us; skew 500 ppm
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:04.779521 6303 init.cc:377] Logged in from keytab as oryx/127.30.71.3@KRBTEST.COM (short username oryx)
I20260810 06:36:04.780045 6303 webserver.cc:460] Webserver started at http://127.30.71.3:46545/ using document root <none> and password file <none>
I20260810 06:36:04.780290 6303 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:04.780366 6303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:04.781641 6303 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 06:36:04.782164 6318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.782343 6303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.782433 6303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "d7677fde3633408daf7b0484fd7e5744"
format_stamp: "Formatted at 2026-08-10 06:36:03 on dist-test-slave-w206"
server_key: "e5a439efba093ea7aa506928068b3e91"
server_key_iv: "12e5df0b965251b73b48fb54e660c204"
server_key_version: "encryptionkey@0"
I20260810 06:36:04.782752 6303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:04.798220 6303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:04.800566 6303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:04.800701 6303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:04.800990 6303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:04.801323 6303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:04.801414 6303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.801458 6303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:04.801478 6303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:04.808054 6303 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:40743
I20260810 06:36:04.808135 6441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:40743 every 8 connection(s)
I20260810 06:36:04.808497 6303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.60@KRBTEST.COM
Aug 10 06:36:04 dist-test-slave-w206 krb5kdc[5066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.61@KRBTEST.COM
I20260810 06:36:04.818209 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 6303
I20260810 06:36:04.820086 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.809331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:47221 (local address 127.30.71.60:41963)
0810 06:36:04.809386 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.809388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.809810 (+ 422us) server_negotiation.cc:407] Connection header received
0810 06:36:04.811131 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.811133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.811144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.811198 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.811863 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.812626 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.813769 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.814101 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.814139 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.816708 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.816709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.816709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.816717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.818493 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.818957 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.818959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.818959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.819051 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819317 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.819318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.819318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.819465 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.819512 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.819860 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.819971 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:36:04.820102 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.809612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:40731 (local address 127.30.71.62:35649)
0810 06:36:04.809665 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.809667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.809992 (+ 325us) server_negotiation.cc:407] Connection header received
0810 06:36:04.811012 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.811013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.811026 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.811059 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.811905 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.812449 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.813451 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.813662 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.813750 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.816379 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.816381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.816381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.816390 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.818263 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.818863 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.818865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.818865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.818930 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819305 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.819306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.819307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.819453 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.819500 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.819861 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.819958 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:36:04.820302 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.809297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:40731)
0810 06:36:04.809917 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.809926 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.810853 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.811139 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.811140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.811545 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.811748 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.811752 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.812558 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.812558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.813353 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.813355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.813761 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.813762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.813897 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.814465 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.814476 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.816240 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.818368 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.818377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.818379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.818770 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.819031 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.819033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.819200 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.819551 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.819558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.819707 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.819822 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.820001 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":102,"threads_started":1}
I20260810 06:36:04.820523 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.809257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.30.71.3:47221)
0810 06:36:04.809564 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.809610 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.810836 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.811228 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.811229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.811620 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.811748 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.811764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.812726 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.812727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.813654 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.813656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.814210 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.814211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.814284 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.814907 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.814912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.816624 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.818603 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.818604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.818605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.818862 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.819131 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.819132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.819221 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.819563 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.819564 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.819713 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.820270 (+ 557us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.820414 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":52,"thread_start_us":125,"threads_started":1}
I20260810 06:36:04.820884 6442 heartbeater.cc:347] Connected to a master server at 127.30.71.60:41963
I20260810 06:36:04.821008 6442 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.821097 6443 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35649
I20260810 06:36:04.821156 6443 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.821237 6442 heartbeater.cc:511] Master 127.30.71.60:41963 requested a full tablet report, sending...
I20260810 06:36:04.821357 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.809534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:60149 (local address 127.30.71.61:35069)
0810 06:36:04.809607 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.809609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.809816 (+ 207us) server_negotiation.cc:407] Connection header received
0810 06:36:04.810963 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.810965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.811002 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.811063 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.811860 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.812632 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.813759 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.814069 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.814112 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.817322 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.817324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.817324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.817333 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.819203 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819668 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.819669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.819669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.819726 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819969 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.819970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.819970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.820096 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.820839 (+ 743us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.821116 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.821265 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:36:04.821237 6443 heartbeater.cc:511] Master 127.30.71.62:35649 requested a full tablet report, sending...
I20260810 06:36:04.821902 5114 ts_manager.cc:194] Re-registered known tserver with Master: d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743)
I20260810 06:36:04.821926 5960 ts_manager.cc:194] Registered new tserver with Master: d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743)
I20260810 06:36:04.822420 5114 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:40731
I20260810 06:36:04.823369 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.809257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.30.71.3:60149)
0810 06:36:04.809596 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.809611 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.810836 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.811133 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.811136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.811619 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:04.811748 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.811756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.812728 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.812728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.813658 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.813659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.814196 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.814197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.814309 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.814815 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:04.814819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:04.817230 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:04.819368 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.819369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.819585 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.819817 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:04.819818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:04.819818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:04.819876 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:04.820897 (+ 1021us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:04.820898 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:04.820951 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.821046 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.821153 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":52,"thread_start_us":115,"threads_started":1}
I20260810 06:36:04.824483 6444 heartbeater.cc:347] Connected to a master server at 127.30.71.61:35069
I20260810 06:36:04.824529 6444 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:04.824595 6444 heartbeater.cc:511] Master 127.30.71.61:35069 requested a full tablet report, sending...
I20260810 06:36:04.824893 5753 ts_manager.cc:194] Registered new tserver with Master: d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743)
I20260810 06:36:04.825261 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.821503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.0.0.1:34504)
0810 06:36:04.821547 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.821559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.821603 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.821906 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.821907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.821909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:04.821954 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.821956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.822902 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.822903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.824127 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.824130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.825062 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.825063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.825126 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.825129 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.825159 (+ 30us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.825196 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 06:36:04.825644 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.821626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.30.71.60:41963)
0810 06:36:04.821744 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.821746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.821753 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:36:04.821774 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.821774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.821808 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.822017 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:04.822050 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.822809 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.824359 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.824969 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.825433 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.825517 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.825536 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.825577 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260810 06:36:04.825858 5970 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 06:36:04.826048 6001 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6c80a34619c4d6a8f4efaa8738de4bb. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.826144 6001 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:36:04.831300 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.821587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.30.71.62:35649)
0810 06:36:04.821817 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.821820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.822740 (+ 920us) server_negotiation.cc:407] Connection header received
0810 06:36:04.823014 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.823015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.823025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.823054 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.823821 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.824564 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.825701 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.825974 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.826011 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.827862 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.827864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.827864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.827872 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.829895 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.830369 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.830370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.830370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.830418 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.830647 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.830648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.830648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.830793 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.830889 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.831103 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.831173 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260810 06:36:04.831532 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.822130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36564 (local address 127.30.71.61:35069)
0810 06:36:04.822166 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.822168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.822756 (+ 588us) server_negotiation.cc:407] Connection header received
0810 06:36:04.822853 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.822854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.822865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.822895 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.823949 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.824617 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.826091 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.826380 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.826421 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.828319 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.828321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.828321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.828328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.830060 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.830669 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.830670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.830670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.830723 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.830971 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.830972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.830972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.831113 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.831178 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.831370 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.831455 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:36:04.831532 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.821366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.30.71.60:41963)
0810 06:36:04.821403 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.821405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.822850 (+ 1445us) server_negotiation.cc:407] Connection header received
0810 06:36:04.823117 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.823118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.823133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.823160 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.825332 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.826027 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.827089 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.827291 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.827345 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.828660 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.828661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.828661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.828668 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.830213 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.830612 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.830613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.830614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.830671 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.830850 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.830851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.830851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.831028 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.831115 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.831334 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.831415 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:36:04.832957 5114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:36:04.834551 5114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:04.835346 5917 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: NON_VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: true } } }
I20260810 06:36:04.835445 5917 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.836704 5917 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: attempting to promote NON_VOTER f2d7a39f822b432996902930410e53f4 to VOTER
I20260810 06:36:04.836827 5917 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:04.837193 5763 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Refusing update from remote peer c6c80a34619c4d6a8f4efaa8738de4bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 06:36:04.837378 5713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 06:36:04.837715 5970 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 LEARNER]: Refusing update from remote peer c6c80a34619c4d6a8f4efaa8738de4bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 06:36:04.838069 5712 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 06:36:04.838074 5713 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2d7a39f822b432996902930410e53f4 (127.30.71.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.838251 5763 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2d7a39f822b432996902930410e53f4 (127.30.71.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.838575 5970 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2d7a39f822b432996902930410e53f4 (127.30.71.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.839141 5713 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.839224 5713 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.839190 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.839231 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.839280 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.839301 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:36:04.839716 6001 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.839809 6001 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:36:04.839964 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:04.840063 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:04.844404 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.840389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:41201 (local address 127.0.0.1:37734)
0810 06:36:04.840429 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.840444 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.840488 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.841083 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.841084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.841086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:04.841134 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.841137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.842460 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.842460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.843502 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.843504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.844180 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.844181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.844238 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.844241 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.844297 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.844335 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 06:36:04.844432 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.840490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37734 (local address 127.30.71.1:41201)
0810 06:36:04.840924 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.840929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.840940 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:04.840981 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.840983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.841001 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.841075 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:04.841222 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.842376 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.843573 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.844078 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.844178 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.844279 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.844310 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.844349 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":98,"threads_started":1}
I20260810 06:36:04.844578 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.840672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:40743 (local address 127.0.0.1:51020)
0810 06:36:04.841125 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.841136 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.841189 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.841375 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.841376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.841378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:04.841418 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.841420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.842245 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.842246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.843435 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.843438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.844349 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.844350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.844412 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.844417 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.844447 (+ 30us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.844485 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260810 06:36:04.844834 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.840434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:33097 (local address 127.0.0.1:60442)
0810 06:36:04.840735 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.840747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.840797 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.841491 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.841493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.841495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:04.841546 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.841548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.842834 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.842835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.843709 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.843712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.844625 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.844626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.844686 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.844689 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.844721 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.844753 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":95,"threads_started":1}
I20260810 06:36:04.844936 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.840786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51020 (local address 127.30.71.3:40743)
0810 06:36:04.841204 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.841208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.841217 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:04.841268 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.841274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.841287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.841377 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:04.841475 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.842140 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.843527 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.844233 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.844678 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.844773 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.844799 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.844843 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":92,"threads_started":1}
I20260810 06:36:04.845288 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.840894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.30.71.2:33097)
0810 06:36:04.841321 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.841327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.841336 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:04.841374 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.841376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.841391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.841616 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:04.841674 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.842694 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.843822 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.844524 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.845050 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.845134 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.845160 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.845203 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":97,"threads_started":1}
I20260810 06:36:04.845242 6081 tablet_service.cc:1467] Processing CreateTablet for tablet 556905b8c28f4834a73dc3d433475e44 (DEFAULT_TABLE table=test-table [id=65490948d554457183133fa475ab810c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:04.845674 6081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556905b8c28f4834a73dc3d433475e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:04.845721 6371 tablet_service.cc:1467] Processing CreateTablet for tablet 556905b8c28f4834a73dc3d433475e44 (DEFAULT_TABLE table=test-table [id=65490948d554457183133fa475ab810c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:04.845827 6227 tablet_service.cc:1467] Processing CreateTablet for tablet 556905b8c28f4834a73dc3d433475e44 (DEFAULT_TABLE table=test-table [id=65490948d554457183133fa475ab810c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:04.846055 6371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556905b8c28f4834a73dc3d433475e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:04.846150 6227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556905b8c28f4834a73dc3d433475e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:04.849633 6472 tablet_bootstrap.cc:492] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Bootstrap starting.
I20260810 06:36:04.849961 6473 tablet_bootstrap.cc:492] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Bootstrap starting.
I20260810 06:36:04.850576 6472 tablet_bootstrap.cc:654] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:04.850783 6473 tablet_bootstrap.cc:654] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:04.851004 6472 log.cc:824] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:04.851136 6474 tablet_bootstrap.cc:492] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Bootstrap starting.
I20260810 06:36:04.851246 6473 log.cc:824] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:04.851912 6474 tablet_bootstrap.cc:654] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:04.852059 6472 tablet_bootstrap.cc:492] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: No bootstrap required, opened a new log
I20260810 06:36:04.852162 6472 ts_tablet_manager.cc:1397] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:36:04.852327 6474 log.cc:824] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:04.853087 6474 tablet_bootstrap.cc:492] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: No bootstrap required, opened a new log
I20260810 06:36:04.853164 6474 ts_tablet_manager.cc:1397] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:36:04.854302 6472 raft_consensus.cc:348] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.854489 6472 raft_consensus.cc:374] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:04.854535 6472 raft_consensus.cc:729] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5813f7c05d041ce8d3ab0673b7db1d0, State: Initialized, Role: FOLLOWER
I20260810 06:36:04.854671 6472 consensus_queue.cc:260] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.855012 6472 ts_tablet_manager.cc:1428] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:36:04.855049 6153 heartbeater.cc:503] Master 127.30.71.62:35649 was elected leader, sending a full tablet report...
I20260810 06:36:04.855232 6474 raft_consensus.cc:348] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.855963 6474 raft_consensus.cc:374] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:04.856012 6474 raft_consensus.cc:729] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f0ddc883dc4e4ea46f3442414adbd4, State: Initialized, Role: FOLLOWER
I20260810 06:36:04.856258 6473 tablet_bootstrap.cc:492] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: No bootstrap required, opened a new log
I20260810 06:36:04.856341 6473 ts_tablet_manager.cc:1397] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260810 06:36:04.856314 6474 consensus_queue.cc:260] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.856626 6296 heartbeater.cc:503] Master 127.30.71.62:35649 was elected leader, sending a full tablet report...
I20260810 06:36:04.856958 6474 ts_tablet_manager.cc:1428] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:36:04.858934 6473 raft_consensus.cc:348] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.859155 6473 raft_consensus.cc:374] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:04.859207 6473 raft_consensus.cc:729] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7677fde3633408daf7b0484fd7e5744, State: Initialized, Role: FOLLOWER
I20260810 06:36:04.859329 6473 consensus_queue.cc:260] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.859663 6473 ts_tablet_manager.cc:1428] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 06:36:04.859716 6443 heartbeater.cc:503] Master 127.30.71.62:35649 was elected leader, sending a full tablet report...
I20260810 06:36:04.894001 6478 raft_consensus.cc:482] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:04.894135 6478 raft_consensus.cc:504] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.894502 6478 leader_election.cc:288] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743), 36f0ddc883dc4e4ea46f3442414adbd4 (127.30.71.2:33097)
I20260810 06:36:04.897363 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.894710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:37433 (local address 127.30.71.3:40743)
0810 06:36:04.894755 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.894757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.894769 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:36:04.894883 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.894884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.894898 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.894965 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:04.895252 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.895712 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.896655 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.897005 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.897090 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.897100 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.897246 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.897284 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:36:04.897398 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.894756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:40293 (local address 127.30.71.2:33097)
0810 06:36:04.894803 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.894804 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.894814 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:04.894875 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.894876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.894894 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.894989 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:04.895163 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.895685 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.896652 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.897024 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.897120 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.897130 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.897285 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.897327 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:36:04.897410 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.894675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:33097 (local address 127.30.71.1:40293)
0810 06:36:04.894749 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.894771 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.894815 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.894996 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.894997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.894998 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:04.895083 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.895085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.895787 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.895788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.896549 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.896551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.897123 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.897124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.897207 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.897210 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.897277 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.897316 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":3}
I20260810 06:36:04.897434 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.894647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:40743 (local address 127.30.71.1:37433)
0810 06:36:04.894726 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:04.894740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:04.894788 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:04.894968 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:04.894969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:04.894995 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:04.895163 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.895165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.895795 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.895796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.896559 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:04.896561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.897088 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.897089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:04.897169 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.897172 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:04.897241 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:36:04.897280 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":6}
I20260810 06:36:04.897774 6249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556905b8c28f4834a73dc3d433475e44" candidate_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" is_pre_election: true
I20260810 06:36:04.897778 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556905b8c28f4834a73dc3d433475e44" candidate_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7677fde3633408daf7b0484fd7e5744" is_pre_election: true
I20260810 06:36:04.897969 6394 raft_consensus.cc:2454] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5813f7c05d041ce8d3ab0673b7db1d0 in term 0.
I20260810 06:36:04.897956 6249 raft_consensus.cc:2454] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5813f7c05d041ce8d3ab0673b7db1d0 in term 0.
I20260810 06:36:04.898239 6031 leader_election.cc:302] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f0ddc883dc4e4ea46f3442414adbd4, c5813f7c05d041ce8d3ab0673b7db1d0; no voters:
I20260810 06:36:04.898458 6478 raft_consensus.cc:2781] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:04.898517 6478 raft_consensus.cc:482] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:04.898542 6478 raft_consensus.cc:3037] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:04.899380 6478 raft_consensus.cc:504] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.899641 6478 leader_election.cc:288] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [CANDIDATE]: Term 1 election: Requested vote from peers d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743), 36f0ddc883dc4e4ea46f3442414adbd4 (127.30.71.2:33097)
I20260810 06:36:04.899720 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556905b8c28f4834a73dc3d433475e44" candidate_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7677fde3633408daf7b0484fd7e5744"
I20260810 06:36:04.899735 6249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556905b8c28f4834a73dc3d433475e44" candidate_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f0ddc883dc4e4ea46f3442414adbd4"
I20260810 06:36:04.899807 6249 raft_consensus.cc:3037] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:04.899809 6394 raft_consensus.cc:3037] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:04.900550 6249 raft_consensus.cc:2454] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5813f7c05d041ce8d3ab0673b7db1d0 in term 1.
I20260810 06:36:04.900732 6031 leader_election.cc:302] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f0ddc883dc4e4ea46f3442414adbd4, c5813f7c05d041ce8d3ab0673b7db1d0; no voters:
I20260810 06:36:04.900856 6478 raft_consensus.cc:2781] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:04.901015 6478 raft_consensus.cc:686] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 1 LEADER]: Becoming Leader. State: Replica: c5813f7c05d041ce8d3ab0673b7db1d0, State: Running, Role: LEADER
I20260810 06:36:04.901109 6478 consensus_queue.cc:237] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } }
I20260810 06:36:04.901429 6394 raft_consensus.cc:2454] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5813f7c05d041ce8d3ab0673b7db1d0 in term 1.
I20260810 06:36:04.902240 5114 catalog_manager.cc:5495] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1). New cstate: current_term: 1 leader_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } health_report { overall_health: UNKNOWN } } }
I20260810 06:36:04.917791 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.915352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.30.71.1:41201)
0810 06:36:04.915425 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.915427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.915434 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:36:04.915477 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.915478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.915497 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.915573 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:04.915733 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.916217 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.916993 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.917192 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.917269 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.917445 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:04.917555 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:04.917656 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.917675 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.917696 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 06:36:04.917951 6298 tablet.cc:2369] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:36:04.920058 6394 raft_consensus.cc:1261] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Refusing update from remote peer c5813f7c05d041ce8d3ab0673b7db1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:04.920058 6249 raft_consensus.cc:1261] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 FOLLOWER]: Refusing update from remote peer c5813f7c05d041ce8d3ab0673b7db1d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:04.920487 6482 consensus_queue.cc:1035] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:04.920656 6478 consensus_queue.cc:1035] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:04.924013 6486 mvcc.cc:205] Tried to move back new op lower bound from 7316864061108940800 to 7316864061035511808. Current Snapshot: MvccSnapshot[applied={T|T < 7316864061108940800}]
I20260810 06:36:04.924110 6485 mvcc.cc:205] Tried to move back new op lower bound from 7316864061108940800 to 7316864061035511808. Current Snapshot: MvccSnapshot[applied={T|T < 7316864061108940800}]
I20260810 06:36:04.925276 6487 mvcc.cc:205] Tried to move back new op lower bound from 7316864061108940800 to 7316864061035511808. Current Snapshot: MvccSnapshot[applied={T|T < 7316864061108940800}]
I20260810 06:36:04.926429 5114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:04.926532 5114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:04.927855 5114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6c80a34619c4d6a8f4efaa8738de4bb: Sending DeleteTablet for 3 replicas of tablet 556905b8c28f4834a73dc3d433475e44
I20260810 06:36:04.928187 6371 tablet_service.cc:1514] Processing DeleteTablet for tablet 556905b8c28f4834a73dc3d433475e44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:04 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:51020
I20260810 06:36:04.928460 6494 tablet_replica.cc:330] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: stopping tablet replica
I20260810 06:36:04.928612 6494 raft_consensus.cc:2229] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:04.928601 6081 tablet_service.cc:1514] Processing DeleteTablet for tablet 556905b8c28f4834a73dc3d433475e44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:04 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:37734
I20260810 06:36:04.928795 6494 raft_consensus.cc:2258] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:04.928849 6227 tablet_service.cc:1514] Processing DeleteTablet for tablet 556905b8c28f4834a73dc3d433475e44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:04 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:60442
I20260810 06:36:04.928913 6495 tablet_replica.cc:330] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: stopping tablet replica
I20260810 06:36:04.928998 6495 raft_consensus.cc:2229] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:36:04.929112 6495 raft_consensus.cc:2258] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:04.929459 6494 ts_tablet_manager.cc:1905] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:04.929486 6495 ts_tablet_manager.cc:1905] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:04.930339 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 5081
I20260810 06:36:04.931370 6496 tablet_replica.cc:330] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: stopping tablet replica
I20260810 06:36:04.931490 6496 raft_consensus.cc:2229] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:04.931530 6495 ts_tablet_manager.cc:1918] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:04.931607 6496 raft_consensus.cc:2258] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:04.931627 6495 log.cc:1196] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/556905b8c28f4834a73dc3d433475e44
I20260810 06:36:04.931716 6494 ts_tablet_manager.cc:1918] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:04.931842 6494 log.cc:1196] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/556905b8c28f4834a73dc3d433475e44
I20260810 06:36:04.932081 6496 ts_tablet_manager.cc:1905] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:04.932195 6495 ts_tablet_manager.cc:1939] T 556905b8c28f4834a73dc3d433475e44 P c5813f7c05d041ce8d3ab0673b7db1d0: Deleting consensus metadata
I20260810 06:36:04.932241 6494 ts_tablet_manager.cc:1939] T 556905b8c28f4834a73dc3d433475e44 P d7677fde3633408daf7b0484fd7e5744: Deleting consensus metadata
I20260810 06:36:04.933508 6496 ts_tablet_manager.cc:1918] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:04.933598 6496 log.cc:1196] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/556905b8c28f4834a73dc3d433475e44
I20260810 06:36:04.933847 6496 ts_tablet_manager.cc:1939] T 556905b8c28f4834a73dc3d433475e44 P 36f0ddc883dc4e4ea46f3442414adbd4: Deleting consensus metadata
W20260810 06:36:04.940945 6319 connection.cc:535] server connection from 127.0.0.1:51020 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 06:36:04.941201 6031 connection.cc:535] server connection from 127.0.0.1:37734 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 06:36:04.941205 6179 connection.cc:535] server connection from 127.0.0.1:60442 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:36:04.942294 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.929867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34520 (local address 127.30.71.60:41963)
0810 06:36:04.929923 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.929925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.935282 (+ 5357us) server_negotiation.cc:407] Connection header received
0810 06:36:04.935361 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.935362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.935373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.935446 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.936070 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.936560 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.937276 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.937470 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.937557 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.939375 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.939376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.939377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.939385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.940850 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.941285 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.941286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.941287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.941347 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.941775 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.941776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.941776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.941886 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.941998 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.942108 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.942190 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:36:04.951510 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.943573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.30.71.61:35069)
0810 06:36:04.943663 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.943666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.944000 (+ 334us) server_negotiation.cc:407] Connection header received
0810 06:36:04.944093 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.944094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.944106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.944138 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.944807 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.945278 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.946385 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.946595 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.946674 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.948217 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.948219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.948219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.948227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.950009 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.950461 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.950462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.950463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.950527 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.950796 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.950797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.950798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.950949 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.951124 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.951325 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.951404 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:36:04.952000 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:04.943583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34522 (local address 127.30.71.60:41963)
0810 06:36:04.943696 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:04.943698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:04.943773 (+ 75us) server_negotiation.cc:407] Connection header received
0810 06:36:04.943846 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:04.943847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:04.943858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:04.943917 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:04.944640 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.945393 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.946365 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:04.946665 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:04.946731 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:04.948318 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:04.948319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:04.948319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:04.948326 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:04.950145 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.950571 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.950572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.950573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.950634 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:04.950912 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:04.950913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:04.950913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:04.951226 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:04.951319 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:04.951505 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:36:04.951943 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 06:36:05.925338 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:05.924757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:48309)
0810 06:36:05.924930 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:05.925141 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260810 06:36:05.925637 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:05.925372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:38605)
0810 06:36:05.925520 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:05.925590 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":53,"threads_started":1}
W20260810 06:36:05.925633 6296 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:35649 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
W20260810 06:36:05.925987 6443 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:35649 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:05.926230 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:05.925968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:49885)
0810 06:36:05.926124 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:05.926197 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":60,"threads_started":1}
W20260810 06:36:05.926465 6153 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:35649 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:06.428108 6508 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6c80a34619c4d6a8f4efaa8738de4bb)
I20260810 06:36:06.428207 6510 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6c80a34619c4d6a8f4efaa8738de4bb)
I20260810 06:36:06.428315 6508 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:06.428357 6510 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:06.428635 6508 leader_election.cc:288] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649), a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61:35069)
I20260810 06:36:06.428882 6510 leader_election.cc:288] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649), f2d7a39f822b432996902930410e53f4 (127.30.71.60:41963)
I20260810 06:36:06.429108 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.428781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57394)
0810 06:36:06.428994 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.429063 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":48,"threads_started":1}
W20260810 06:36:06.429302 5943 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:06.429790 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.429702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57398)
0810 06:36:06.429743 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.429764 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 06:36:06.429867 5943 leader_election.cc:334] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649): Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:06.429961 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.428932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57396)
0810 06:36:06.429556 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.429921 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":42,"threads_started":1}
W20260810 06:36:06.430061 5738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:06.430500 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.430390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57408)
0810 06:36:06.430445 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.430476 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:36:06.430635 5738 leader_election.cc:334] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649): Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:06.432152 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.428829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:35069 (local address 127.0.0.1:41388)
0810 06:36:06.429018 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.429029 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:06.429090 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:06.429291 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:06.429292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:06.429295 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:06.429356 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:06.429359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.430308 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.430309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:06.431324 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:06.431327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.431888 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.431889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:06.431959 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:06.431963 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:36:06.432036 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:36:06.432081 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":66,"threads_started":1}
I20260810 06:36:06.432206 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.429053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.30.71.61:35069)
0810 06:36:06.429131 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:06.429133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:06.429141 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:06.429165 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:06.429166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:06.429176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:06.429249 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:06.429456 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.430233 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.431411 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.431786 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.431884 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:06.431897 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:06.432045 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 06:36:06.432089 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":64,"thread_start_us":41,"threads_started":1}
I20260810 06:36:06.432360 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.428977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.30.71.60:41963)
0810 06:36:06.429434 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:06.429436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:06.429444 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:06.429466 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:06.429468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:06.429481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:06.429565 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:06.429752 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.430516 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.431627 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.432012 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.432101 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:06.432112 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:06.432264 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 06:36:06.432311 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":437,"thread_start_us":42,"threads_started":1}
I20260810 06:36:06.432420 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.428802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:41963 (local address 127.0.0.1:57360)
0810 06:36:06.429063 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.429077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:06.429142 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:06.429612 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:06.429613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:06.429616 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:06.429667 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:06.429669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.430635 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.430636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:06.431536 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:06.431539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:06.432104 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:06.432105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:06.432169 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:06.432172 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:06.432249 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:36:06.432303 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":2,"thread_start_us":42,"threads_started":1}
I20260810 06:36:06.432711 5970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0f32c2e86ec416e82d5fb893254be3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2d7a39f822b432996902930410e53f4" is_pre_election: true
I20260810 06:36:06.432701 5763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d7a39f822b432996902930410e53f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a0f32c2e86ec416e82d5fb893254be3b" is_pre_election: true
I20260810 06:36:06.432821 5970 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0f32c2e86ec416e82d5fb893254be3b in term 1.
I20260810 06:36:06.432821 5763 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2d7a39f822b432996902930410e53f4 in term 1.
I20260810 06:36:06.432996 5944 leader_election.cc:302] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0f32c2e86ec416e82d5fb893254be3b, f2d7a39f822b432996902930410e53f4; no voters: c6c80a34619c4d6a8f4efaa8738de4bb
I20260810 06:36:06.433001 5737 leader_election.cc:302] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0f32c2e86ec416e82d5fb893254be3b, f2d7a39f822b432996902930410e53f4; no voters: c6c80a34619c4d6a8f4efaa8738de4bb
I20260810 06:36:06.433112 6508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:36:06.433123 6510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:36:06.433171 6510 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c6c80a34619c4d6a8f4efaa8738de4bb)
I20260810 06:36:06.433169 6508 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c6c80a34619c4d6a8f4efaa8738de4bb)
I20260810 06:36:06.433194 6510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:36:06.433230 6508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:36:06.434043 6510 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:06.434119 6508 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:06.434224 6510 leader_election.cc:288] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [CANDIDATE]: Term 2 election: Requested vote from peers c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649), f2d7a39f822b432996902930410e53f4 (127.30.71.60:41963)
I20260810 06:36:06.434280 6508 leader_election.cc:288] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 2 election: Requested vote from peers c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649), a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61:35069)
I20260810 06:36:06.434420 5763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d7a39f822b432996902930410e53f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a0f32c2e86ec416e82d5fb893254be3b"
I20260810 06:36:06.434504 5763 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2d7a39f822b432996902930410e53f4 in current term 2: Already voted for candidate a0f32c2e86ec416e82d5fb893254be3b in this term.
I20260810 06:36:06.434501 5970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0f32c2e86ec416e82d5fb893254be3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2d7a39f822b432996902930410e53f4"
I20260810 06:36:06.434557 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.434467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57430)
0810 06:36:06.434501 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.434533 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 06:36:06.434584 5970 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a0f32c2e86ec416e82d5fb893254be3b in current term 2: Already voted for candidate f2d7a39f822b432996902930410e53f4 in this term.
I20260810 06:36:06.434815 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.434380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57416)
0810 06:36:06.434761 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.434796 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366}
I20260810 06:36:06.435000 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.434889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57438)
0810 06:36:06.434924 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.434954 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260810 06:36:06.435062 5738 leader_election.cc:334] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649): Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:06.435130 5738 leader_election.cc:302] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0f32c2e86ec416e82d5fb893254be3b; no voters: c6c80a34619c4d6a8f4efaa8738de4bb, f2d7a39f822b432996902930410e53f4
I20260810 06:36:06.435226 6510 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 06:36:06.435286 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.435205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57442)
0810 06:36:06.435239 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.435262 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260810 06:36:06.435374 5943 leader_election.cc:334] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649): Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:06.435413 5943 leader_election.cc:302] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2d7a39f822b432996902930410e53f4; no voters: a0f32c2e86ec416e82d5fb893254be3b, c6c80a34619c4d6a8f4efaa8738de4bb
I20260810 06:36:06.435498 6508 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 06:36:06.926584 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.926081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:56839)
0810 06:36:06.926478 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.926527 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":88,"threads_started":1}
I20260810 06:36:06.926581 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.926285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:57109)
0810 06:36:06.926463 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.926524 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":67,"threads_started":1}
I20260810 06:36:06.927052 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:06.926809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:37333)
0810 06:36:06.926933 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:06.926959 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260810 06:36:07.927534 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:07.927095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:47783)
0810 06:36:07.927428 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:07.927473 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":71,"threads_started":1}
I20260810 06:36:07.927716 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:07.927150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:40251)
0810 06:36:07.927611 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:07.927656 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":66,"threads_started":1}
I20260810 06:36:07.927829 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:07.927524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:53421)
0810 06:36:07.927731 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:07.927781 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":49,"threads_started":1}
I20260810 06:36:07.954699 6527 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:07.954823 6527 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:07.955039 6527 leader_election.cc:288] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649), a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61:35069)
I20260810 06:36:07.955266 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:07.955109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57674)
0810 06:36:07.955191 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:07.955225 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65,"thread_start_us":50,"threads_started":1}
I20260810 06:36:07.955297 5763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d7a39f822b432996902930410e53f4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a0f32c2e86ec416e82d5fb893254be3b" is_pre_election: true
I20260810 06:36:07.955442 5763 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2d7a39f822b432996902930410e53f4 in term 2.
I20260810 06:36:07.955636 5944 leader_election.cc:302] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0f32c2e86ec416e82d5fb893254be3b, f2d7a39f822b432996902930410e53f4; no voters:
I20260810 06:36:07.955688 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:07.955577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57682)
0810 06:36:07.955635 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:07.955665 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 06:36:07.955755 6527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260810 06:36:07.955796 5943 leader_election.cc:334] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649): Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:07.955813 6527 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:07.955838 6527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:36:07.956665 6527 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:07.956815 6527 leader_election.cc:288] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 3 election: Requested vote from peers c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649), a0f32c2e86ec416e82d5fb893254be3b (127.30.71.61:35069)
I20260810 06:36:07.956986 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:07.956884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57688)
0810 06:36:07.956933 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:07.956959 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 06:36:07.957007 5763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d7a39f822b432996902930410e53f4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a0f32c2e86ec416e82d5fb893254be3b"
I20260810 06:36:07.957108 5763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:36:07.957361 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:07.957258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57692)
0810 06:36:07.957312 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:07.957339 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:36:07.957473 5943 leader_election.cc:334] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649): Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
I20260810 06:36:07.957929 5763 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d7a39f822b432996902930410e53f4 in term 3.
I20260810 06:36:07.958092 5944 leader_election.cc:302] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0f32c2e86ec416e82d5fb893254be3b, f2d7a39f822b432996902930410e53f4; no voters: c6c80a34619c4d6a8f4efaa8738de4bb
I20260810 06:36:07.958194 6527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:36:07.958312 6527 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [term 3 LEADER]: Becoming Leader. State: Replica: f2d7a39f822b432996902930410e53f4, State: Running, Role: LEADER
I20260810 06:36:07.958403 6527 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } }
I20260810 06:36:07.958786 6531 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d7a39f822b432996902930410e53f4. Latest consensus state: current_term: 3 leader_uuid: "f2d7a39f822b432996902930410e53f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:07.958882 6531 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:07.959091 6533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:07.959673 6533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:07.959796 6533 catalog_manager.cc:1259] Loaded cluster ID: 004cb194534c42039442ee686847c45b
I20260810 06:36:07.959836 6533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:36:07.960052 6533 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:07.960162 6533 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Loaded TSK: 0
I20260810 06:36:07.960350 6533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:36:08.042944 5959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:36:08.044705 5959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:08.045897 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.045760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57720)
0810 06:36:08.045825 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.045869 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:36:08.045959 5763 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [term 3 FOLLOWER]: Refusing update from remote peer f2d7a39f822b432996902930410e53f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 06:36:08.046339 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.046162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.0.0.1:57724)
0810 06:36:08.046230 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.046261 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:36:08.046238 6531 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 06:36:08.046445 5943 consensus_peers.cc:487] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 -> Peer c6c80a34619c4d6a8f4efaa8738de4bb (127.30.71.62:35649): Couldn't send request to peer c6c80a34619c4d6a8f4efaa8738de4bb. Status: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:36:08.047027 6535 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d7a39f822b432996902930410e53f4. Latest consensus state: current_term: 3 leader_uuid: "f2d7a39f822b432996902930410e53f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:08.047145 6535 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:36:08.047715 6534 mvcc.cc:205] Tried to move back new op lower bound from 7316864073913913344 to 7316864073558179840. Current Snapshot: MvccSnapshot[applied={T|T < 7316864073913913344}]
I20260810 06:36:08.047799 6527 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f2d7a39f822b432996902930410e53f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:08.047902 6527 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:08.047895 6536 mvcc.cc:205] Tried to move back new op lower bound from 7316864073913913344 to 7316864073558179840. Current Snapshot: MvccSnapshot[applied={T|T < 7316864073913913344}]
I20260810 06:36:08.048064 6535 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f2d7a39f822b432996902930410e53f4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c6c80a34619c4d6a8f4efaa8738de4bb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35649 } } peers { permanent_uuid: "a0f32c2e86ec416e82d5fb893254be3b" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 35069 } attrs { promote: false } } peers { permanent_uuid: "f2d7a39f822b432996902930410e53f4" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 41963 } attrs { promote: false } } }
I20260810 06:36:08.048156 6535 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f32c2e86ec416e82d5fb893254be3b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:36:08.053939 6543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.050319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:41201 (local address 127.0.0.1:50808)
0810 06:36:08.050614 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.050625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:08.050668 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:08.050856 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:08.050857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:08.050858 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:08.050890 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.050892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.051881 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.051882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.052856 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.052858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.053672 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.053738 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.053743 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:36:08.053806 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:36:08.053851 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":92,"threads_started":1}
I20260810 06:36:08.054044 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.050334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.30.71.3:40743)
0810 06:36:08.050775 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:08.050778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:08.050786 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:08.050814 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:08.050815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:08.050827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:08.050895 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:08.051079 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.051645 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053103 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.053661 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053731 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.053744 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:08.053917 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 06:36:08.053966 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":77,"threads_started":1}
I20260810 06:36:08.054275 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.050533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50808 (local address 127.30.71.1:41201)
0810 06:36:08.050660 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:08.050662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:08.050671 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:08.050736 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:08.050737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:08.050749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:08.050813 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:08.051004 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.051779 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.052947 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.053573 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053653 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.053668 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:08.054155 (+ 487us) server_negotiation.cc:299] Negotiation successful
0810 06:36:08.054206 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260810 06:36:08.054318 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.050511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.30.71.2:33097)
0810 06:36:08.050619 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:08.050621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:08.050632 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:08.050662 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:08.050662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:08.050675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:08.050962 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:08.051053 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.051796 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053086 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.053654 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053771 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.053785 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:08.053975 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 06:36:08.054030 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":49,"threads_started":1}
I20260810 06:36:08.054041 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.050233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:33097 (local address 127.0.0.1:34278)
0810 06:36:08.050267 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.050279 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:08.050331 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:08.050791 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:08.050793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:08.050795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:08.050857 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.050859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.051867 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.051867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.052995 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.052998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.053745 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.053837 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.053840 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:08.053889 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:36:08.053926 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":27}
I20260810 06:36:08.054167 6541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.050241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:40743 (local address 127.0.0.1:49302)
0810 06:36:08.050418 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.050426 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:08.050464 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:08.050943 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:08.050944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:08.050946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:08.051022 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.051024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.051714 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.051715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.052995 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.052998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.053744 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.053745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.053830 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.053834 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:36:08.053896 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 06:36:08.053949 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":102,"thread_start_us":83,"threads_started":1}
I20260810 06:36:08.054632 6227 tablet_service.cc:1467] Processing CreateTablet for tablet 68727a9f931d413a86b5ac30ecd2a249 (DEFAULT_TABLE table=test-table [id=12e01e71aeac4db79dd2ab32b1a757fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:08.054685 6081 tablet_service.cc:1467] Processing CreateTablet for tablet 68727a9f931d413a86b5ac30ecd2a249 (DEFAULT_TABLE table=test-table [id=12e01e71aeac4db79dd2ab32b1a757fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:08.054752 6371 tablet_service.cc:1467] Processing CreateTablet for tablet 68727a9f931d413a86b5ac30ecd2a249 (DEFAULT_TABLE table=test-table [id=12e01e71aeac4db79dd2ab32b1a757fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:08.054782 6227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68727a9f931d413a86b5ac30ecd2a249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:08.054795 6081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68727a9f931d413a86b5ac30ecd2a249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:08.054869 6371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68727a9f931d413a86b5ac30ecd2a249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:08.056025 6548 tablet_bootstrap.cc:492] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Bootstrap starting.
I20260810 06:36:08.056298 6549 tablet_bootstrap.cc:492] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0: Bootstrap starting.
I20260810 06:36:08.056502 6548 tablet_bootstrap.cc:654] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:08.056743 6549 tablet_bootstrap.cc:654] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:08.057029 6550 tablet_bootstrap.cc:492] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4: Bootstrap starting.
I20260810 06:36:08.057154 6548 tablet_bootstrap.cc:492] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: No bootstrap required, opened a new log
I20260810 06:36:08.057202 6548 ts_tablet_manager.cc:1397] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:08.057286 6549 tablet_bootstrap.cc:492] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0: No bootstrap required, opened a new log
I20260810 06:36:08.057338 6549 ts_tablet_manager.cc:1397] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:08.057443 6548 raft_consensus.cc:348] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.057516 6548 raft_consensus.cc:374] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:08.057531 6548 raft_consensus.cc:729] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7677fde3633408daf7b0484fd7e5744, State: Initialized, Role: FOLLOWER
I20260810 06:36:08.057468 6549 raft_consensus.cc:348] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.057551 6549 raft_consensus.cc:374] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:08.057585 6548 consensus_queue.cc:260] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.057608 6549 raft_consensus.cc:729] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5813f7c05d041ce8d3ab0673b7db1d0, State: Initialized, Role: FOLLOWER
I20260810 06:36:08.057664 6549 consensus_queue.cc:260] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.057803 6548 ts_tablet_manager.cc:1428] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:08.057847 6549 ts_tablet_manager.cc:1428] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:08.058166 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.058055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:41585)
0810 06:36:08.058095 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.058141 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 06:36:08.058398 6550 tablet_bootstrap.cc:654] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:08.058462 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.058102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:52793)
0810 06:36:08.058396 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.058431 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271}
I20260810 06:36:08.058804 6442 heartbeater.cc:503] Master 127.30.71.60:41963 was elected leader, sending a full tablet report...
I20260810 06:36:08.059005 6550 tablet_bootstrap.cc:492] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4: No bootstrap required, opened a new log
I20260810 06:36:08.059063 6550 ts_tablet_manager.cc:1397] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:08.059319 6152 heartbeater.cc:503] Master 127.30.71.60:41963 was elected leader, sending a full tablet report...
I20260810 06:36:08.059358 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.059272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:60863)
0810 06:36:08.059309 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.059332 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 06:36:08.059397 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.059272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:56465)
0810 06:36:08.059309 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.059332 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 06:36:08.059562 6550 raft_consensus.cc:348] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.059800 6550 raft_consensus.cc:374] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:08.059865 6550 raft_consensus.cc:729] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f0ddc883dc4e4ea46f3442414adbd4, State: Initialized, Role: FOLLOWER
I20260810 06:36:08.059957 6550 consensus_queue.cc:260] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.060137 6550 ts_tablet_manager.cc:1428] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:08.060417 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.060309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:58035)
0810 06:36:08.060357 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.060389 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 06:36:08.060609 6295 heartbeater.cc:503] Master 127.30.71.60:41963 was elected leader, sending a full tablet report...
I20260810 06:36:08.060715 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.060621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:34077)
0810 06:36:08.060660 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.060684 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 06:36:08.062182 6445 tablet.cc:2369] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:36:08.078287 6556 raft_consensus.cc:482] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:08.078389 6556 raft_consensus.cc:504] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.079099 6556 leader_election.cc:288] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743), c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1:41201)
I20260810 06:36:08.081826 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.079119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:55065 (local address 127.30.71.1:41201)
0810 06:36:08.079170 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:08.079172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:08.079249 (+ 77us) server_negotiation.cc:407] Connection header received
0810 06:36:08.079331 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:08.079333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:08.079347 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:08.079398 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:08.079541 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.080037 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.081080 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.081455 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.081546 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.081559 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:08.081717 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:36:08.081765 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:36:08.081823 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.079054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:41201 (local address 127.30.71.2:55065)
0810 06:36:08.079166 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.079179 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:08.079246 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:08.079401 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:08.079402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:08.079405 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:08.079462 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.079463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.080133 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.080134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.080976 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.080979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.081547 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.081548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.081616 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.081621 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:36:08.081708 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:36:08.081747 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":39}
I20260810 06:36:08.081894 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.079152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:57793 (local address 127.30.71.3:40743)
0810 06:36:08.079206 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:08.079208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:08.079406 (+ 198us) server_negotiation.cc:407] Connection header received
0810 06:36:08.079477 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:08.079478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:08.079488 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:08.079570 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:08.079721 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.080201 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.081129 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.081491 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.081588 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.081600 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:08.081774 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:36:08.081812 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:36:08.082011 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.079103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:40743 (local address 127.30.71.2:57793)
0810 06:36:08.079307 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.079316 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:08.079421 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:08.079575 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:08.079576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:08.079578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:08.079641 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.079643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.080291 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.080292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.081031 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:08.081033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.081619 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.081620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:08.081698 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.081701 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:08.081768 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:36:08.081810 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":129,"threads_started":1}
I20260810 06:36:08.082111 6105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68727a9f931d413a86b5ac30ecd2a249" candidate_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" is_pre_election: true
I20260810 06:36:08.082270 6105 raft_consensus.cc:2454] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36f0ddc883dc4e4ea46f3442414adbd4 in term 0.
I20260810 06:36:08.082255 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68727a9f931d413a86b5ac30ecd2a249" candidate_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7677fde3633408daf7b0484fd7e5744" is_pre_election: true
I20260810 06:36:08.082350 6394 raft_consensus.cc:2454] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36f0ddc883dc4e4ea46f3442414adbd4 in term 0.
I20260810 06:36:08.082492 6179 leader_election.cc:302] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f0ddc883dc4e4ea46f3442414adbd4, c5813f7c05d041ce8d3ab0673b7db1d0; no voters:
I20260810 06:36:08.082654 6556 raft_consensus.cc:2781] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:08.082695 6556 raft_consensus.cc:482] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:08.082710 6556 raft_consensus.cc:3037] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:08.083526 6556 raft_consensus.cc:504] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.083676 6556 leader_election.cc:288] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [CANDIDATE]: Term 1 election: Requested vote from peers d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743), c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1:41201)
I20260810 06:36:08.083834 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68727a9f931d413a86b5ac30ecd2a249" candidate_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7677fde3633408daf7b0484fd7e5744"
I20260810 06:36:08.083925 6394 raft_consensus.cc:3037] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:08.083962 6105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68727a9f931d413a86b5ac30ecd2a249" candidate_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0"
I20260810 06:36:08.084059 6105 raft_consensus.cc:3037] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:08.084553 6394 raft_consensus.cc:2454] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36f0ddc883dc4e4ea46f3442414adbd4 in term 1.
I20260810 06:36:08.084751 6180 leader_election.cc:302] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f0ddc883dc4e4ea46f3442414adbd4, d7677fde3633408daf7b0484fd7e5744; no voters:
I20260810 06:36:08.084853 6556 raft_consensus.cc:2781] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:08.084955 6105 raft_consensus.cc:2454] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36f0ddc883dc4e4ea46f3442414adbd4 in term 1.
I20260810 06:36:08.085057 6556 raft_consensus.cc:686] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 LEADER]: Becoming Leader. State: Replica: 36f0ddc883dc4e4ea46f3442414adbd4, State: Running, Role: LEADER
I20260810 06:36:08.085246 6556 consensus_queue.cc:237] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } }
I20260810 06:36:08.085436 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.085290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:54227)
0810 06:36:08.085348 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.085406 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 06:36:08.086122 5959 catalog_manager.cc:5495] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36f0ddc883dc4e4ea46f3442414adbd4 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f0ddc883dc4e4ea46f3442414adbd4" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 } health_report { overall_health: UNKNOWN } } }
I20260810 06:36:08.101380 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.098742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.30.71.2:33097)
0810 06:36:08.098811 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:08.098814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:08.098823 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:08.098873 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:08.098874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:08.098892 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:08.099029 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:08.099129 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.099663 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.100533 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:08.100732 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:08.100805 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:08.101026 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:08.101137 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:08.101219 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:08.101266 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 06:36:08.101289 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:36:08.102643 6394 raft_consensus.cc:1261] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Refusing update from remote peer 36f0ddc883dc4e4ea46f3442414adbd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:08.102697 6105 raft_consensus.cc:1261] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0 [term 1 FOLLOWER]: Refusing update from remote peer 36f0ddc883dc4e4ea46f3442414adbd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:08.103010 6559 consensus_queue.cc:1035] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7677fde3633408daf7b0484fd7e5744" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:08.103199 6556 consensus_queue.cc:1035] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5813f7c05d041ce8d3ab0673b7db1d0" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 41201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:08.103765 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.103624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:39153)
0810 06:36:08.103698 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.103733 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:36:08.104800 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.104681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:55613)
0810 06:36:08.104732 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.104766 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 06:36:08.105374 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.104875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:33111)
0810 06:36:08.105305 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.105344 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412}
I20260810 06:36:08.105485 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.105381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.3:44595)
0810 06:36:08.105432 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.105457 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:36:08.107036 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.106911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.1:44353)
0810 06:36:08.106947 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.107006 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 06:36:08.107046 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.106136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35649 (local address 127.30.71.2:52817)
0810 06:36:08.106956 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.107019 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:35649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":804}
I20260810 06:36:08.107419 5958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:08.107515 5958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:08.108935 5958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2d7a39f822b432996902930410e53f4: Sending DeleteTablet for 3 replicas of tablet 68727a9f931d413a86b5ac30ecd2a249
I20260810 06:36:08.109268 6371 tablet_service.cc:1514] Processing DeleteTablet for tablet 68727a9f931d413a86b5ac30ecd2a249 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:08 UTC) from {username='oryx', principal='oryx/127.30.71.60@KRBTEST.COM'} at 127.0.0.1:49302
I20260810 06:36:08.109364 6227 tablet_service.cc:1514] Processing DeleteTablet for tablet 68727a9f931d413a86b5ac30ecd2a249 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:08 UTC) from {username='oryx', principal='oryx/127.30.71.60@KRBTEST.COM'} at 127.0.0.1:34278
I20260810 06:36:08.109490 6081 tablet_service.cc:1514] Processing DeleteTablet for tablet 68727a9f931d413a86b5ac30ecd2a249 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:08 UTC) from {username='oryx', principal='oryx/127.30.71.60@KRBTEST.COM'} at 127.0.0.1:50808
I20260810 06:36:08.109805 6573 tablet_replica.cc:330] T 68727a9f931d413a86b5ac30ecd2a249 P c5813f7c05d041ce8d3ab0673b7db1d0: stopping tablet replica
I20260810 06:36:08.109851 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 5914
I20260810 06:36:08.112825 6571 tablet_replica.cc:330] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: stopping tablet replica
I20260810 06:36:08.112942 6571 raft_consensus.cc:2229] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:08.113030 6571 raft_consensus.cc:2258] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:08.113538 6571 ts_tablet_manager.cc:1905] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:08.115403 6571 ts_tablet_manager.cc:1918] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:08.115513 6571 log.cc:1196] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/68727a9f931d413a86b5ac30ecd2a249
I20260810 06:36:08.115839 6571 ts_tablet_manager.cc:1939] T 68727a9f931d413a86b5ac30ecd2a249 P d7677fde3633408daf7b0484fd7e5744: Deleting consensus metadata
I20260810 06:36:08.116264 5945 catalog_manager.cc:4841] TS d7677fde3633408daf7b0484fd7e5744 (127.30.71.3:40743): tablet 68727a9f931d413a86b5ac30ecd2a249 (table test-table [id=12e01e71aeac4db79dd2ab32b1a757fa]) successfully deleted
I20260810 06:36:08.119837 6572 tablet_replica.cc:330] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4: stopping tablet replica
I20260810 06:36:08.120021 6572 raft_consensus.cc:2229] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:36:08.120203 6572 raft_consensus.cc:2258] T 68727a9f931d413a86b5ac30ecd2a249 P 36f0ddc883dc4e4ea46f3442414adbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 06:36:08.120270 6543 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:08.119982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:41201 (local address 127.0.0.1:50822)
0810 06:36:08.120038 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.120049 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:08.120090 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:08.120209 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:41201: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:36:08.120410 5944 catalog_manager.cc:4584] TS c5813f7c05d041ce8d3ab0673b7db1d0 (127.30.71.1:41201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68727a9f931d413a86b5ac30ecd2a249: Network error: Client connection negotiation failed: client connection to 127.30.71.1:41201: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:36:08.120635 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 6161
I20260810 06:36:08.133459 6543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:08.133300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:33097 (local address 127.0.0.1:34298)
0810 06:36:08.133365 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:08.133402 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.2:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 06:36:08.133533 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 6303
I20260810 06:36:08.143839 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 5609
I20260810 06:36:08.152066 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 5813
2026-08-10T06:36:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6217 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:36:08 dist-test-slave-w206 krb5kdc[6577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:36:08 dist-test-slave-w206 krb5kdc[6577](info): set up 2 sockets
Aug 10 06:36:08 dist-test-slave-w206 krb5kdc[6577](info): commencing operation
krb5kdc: starting...
W20260810 06:36:09.264801 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:36:09 dist-test-slave-w206 krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:36:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab.
Entry for principal oryx/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab.
I20260810 06:36:09.395850 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41073
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab
--principal=oryx/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:41073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:09.481580 6590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:09.481825 6590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:09.481850 6590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:09.483134 6590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:09.483180 6590 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:36:09.483196 6590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:09.483210 6590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:09.483223 6590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:09.483237 6590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:09.484929 6590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:41073
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.62.keytab
--principal=oryx/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:09.485224 6590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:09.485491 6590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:09.490665 6596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:09.490677 6595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:09.490689 6598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:09.490788 6590 server_base.cc:1034] running on GCE node
I20260810 06:36:09.491123 6590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:09.491448 6590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:09.492622 6590 hybrid_clock.cc:648] HybridClock initialized: now 1786343769492600 us; error 54 us; skew 500 ppm
Aug 10 06:36:09 dist-test-slave-w206 krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:09.495364 6590 init.cc:377] Logged in from keytab as oryx/127.30.71.62@KRBTEST.COM (short username oryx)
I20260810 06:36:09.495879 6590 webserver.cc:460] Webserver started at http://127.30.71.62:41351/ using document root <none> and password file <none>
I20260810 06:36:09.496136 6590 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:09.496208 6590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:09.496358 6590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:09.497452 6590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "75ce6880f13c4dd1a38d9154c159ef60"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "fb028469499aa831c920ea9d401c68ac"
server_key_iv: "9f3b16c30a411830f21a57cba43ab98a"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.497834 6590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "75ce6880f13c4dd1a38d9154c159ef60"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "fb028469499aa831c920ea9d401c68ac"
server_key_iv: "9f3b16c30a411830f21a57cba43ab98a"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.499347 6590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:36:09.500231 6605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:09.500486 6590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:09.500615 6590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "75ce6880f13c4dd1a38d9154c159ef60"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "fb028469499aa831c920ea9d401c68ac"
server_key_iv: "9f3b16c30a411830f21a57cba43ab98a"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.500715 6590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:09.521129 6590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:09.523327 6590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:09.523509 6590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:09.528353 6590 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:41073
I20260810 06:36:09.528411 6672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:41073 every 8 connection(s)
I20260810 06:36:09.528788 6590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:09.529500 6674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:09.531904 6674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Bootstrap starting.
I20260810 06:36:09.532485 6674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:09.532739 6674 log.cc:824] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:09.533469 6674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: No bootstrap required, opened a new log
I20260810 06:36:09.535204 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 6590
I20260810 06:36:09.535336 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:36:09.535534 31004 external_mini_cluster.cc:1428] Setting key d128ae4363b0821be30ac0b76a364286
I20260810 06:36:09.535884 6674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ce6880f13c4dd1a38d9154c159ef60" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41073 } }
I20260810 06:36:09.536098 6674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:09.536154 6674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ce6880f13c4dd1a38d9154c159ef60, State: Initialized, Role: FOLLOWER
I20260810 06:36:09.536286 6674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ce6880f13c4dd1a38d9154c159ef60" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41073 } }
I20260810 06:36:09.536387 6674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:09.536422 6674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:09.536468 6674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:09.537168 6674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ce6880f13c4dd1a38d9154c159ef60" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41073 } }
I20260810 06:36:09.537262 6674 leader_election.cc:302] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75ce6880f13c4dd1a38d9154c159ef60; no voters:
I20260810 06:36:09.537472 6674 leader_election.cc:288] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:36:09.537513 6679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:09.537633 6679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [term 1 LEADER]: Becoming Leader. State: Replica: 75ce6880f13c4dd1a38d9154c159ef60, State: Running, Role: LEADER
I20260810 06:36:09.537714 6679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ce6880f13c4dd1a38d9154c159ef60" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41073 } }
I20260810 06:36:09.537744 6674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:09.538080 6679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75ce6880f13c4dd1a38d9154c159ef60. Latest consensus state: current_term: 1 leader_uuid: "75ce6880f13c4dd1a38d9154c159ef60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ce6880f13c4dd1a38d9154c159ef60" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41073 } } }
I20260810 06:36:09.538168 6679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:09.538090 6681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75ce6880f13c4dd1a38d9154c159ef60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ce6880f13c4dd1a38d9154c159ef60" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41073 } } }
I20260810 06:36:09.538337 6681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60 [sys.catalog]: This master's current role is: LEADER
W20260810 06:36:09.539719 6696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:36:09.539783 6696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:36:09.539839 6685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:09.540004 6685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:09.541591 6685 catalog_manager.cc:1347] Generated new cluster ID: 84dc03a6d888468d80c21789f8ad6c09
I20260810 06:36:09.541652 6685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:36:09 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:09.546620 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.536114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.30.71.62:41073)
0810 06:36:09.536334 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:09.536344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:09.536380 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:36:09.536513 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:09.536521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:09.536580 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:09.536723 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:09.537403 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.538796 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.539774 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.540360 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.540490 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.543239 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:09.543243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:09.543243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:09.543267 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:09.544993 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.545400 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.545403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.545403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.545510 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.545746 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.545747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.545747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.546010 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:09.546189 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:09.546271 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:36:09.546421 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260810 06:36:09.553372 6685 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:36:09.554086 6685 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:09.560604 6685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Generated new TSK 0
I20260810 06:36:09.560832 6685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab.
Entry for principal oryx/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab.
I20260810 06:36:09.596246 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41073
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab
--principal=oryx/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:09.680229 6701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:09.680445 6701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:09.680472 6701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:09.681663 6701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:09.681710 6701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:09.681732 6701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:36:09.681751 6701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:09.683370 6701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab
--principal=oryx/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:41073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:09.683637 6701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:09.683897 6701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:09.689213 6707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:09.689239 6709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:09.689306 6706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:09.689319 6701 server_base.cc:1034] running on GCE node
I20260810 06:36:09.689668 6701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:09.689924 6701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:09.691052 6701 hybrid_clock.cc:648] HybridClock initialized: now 1786343769691039 us; error 30 us; skew 500 ppm
Aug 10 06:36:09 dist-test-slave-w206 krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:09.693778 6701 init.cc:377] Logged in from keytab as oryx/127.30.71.1@KRBTEST.COM (short username oryx)
I20260810 06:36:09.694245 6701 webserver.cc:460] Webserver started at http://127.30.71.1:38087/ using document root <none> and password file <none>
I20260810 06:36:09.694506 6701 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:09.694579 6701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:09.694739 6701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:09.695935 6701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "b26bcda9d15344f9a6f472d1e0edb153"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "7afb30f8aa8f901cbdd1285003f82588"
server_key_iv: "b8b51dd9d936b6411f73445edad043c1"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.696339 6701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "b26bcda9d15344f9a6f472d1e0edb153"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "7afb30f8aa8f901cbdd1285003f82588"
server_key_iv: "b8b51dd9d936b6411f73445edad043c1"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.697744 6701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:36:09.698554 6716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:09.698731 6701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:09.698837 6701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "b26bcda9d15344f9a6f472d1e0edb153"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "7afb30f8aa8f901cbdd1285003f82588"
server_key_iv: "b8b51dd9d936b6411f73445edad043c1"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.698915 6701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:09.713706 6701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:09.715894 6701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:09.716087 6701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:09.716689 6701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:09.716759 6701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:09.716800 6701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:09.716861 6701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:09.724370 6701 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:39533
Aug 10 06:36:09 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:09.724494 6839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:39533 every 8 connection(s)
I20260810 06:36:09.724812 6701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:09.727953 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.717381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:42829 (local address 127.30.71.62:41073)
0810 06:36:09.717429 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:09.717431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:09.717698 (+ 267us) server_negotiation.cc:407] Connection header received
0810 06:36:09.718247 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:09.718248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:09.718266 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:09.718316 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:09.719201 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.720065 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.721679 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.722132 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.722236 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.724945 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:09.724947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:09.724947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:09.724958 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:09.726485 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.726930 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.726931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.726932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.727020 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.727271 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.727272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.727272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.727398 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:09.727483 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:09.727766 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:36:09.727860 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:36:09.728152 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.717394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41073 (local address 127.30.71.1:42829)
0810 06:36:09.717573 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:09.717610 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:09.718130 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:09.718395 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:09.718399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:09.718799 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:09.719094 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.719100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.720162 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.720163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.721554 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.721558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.722235 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.722236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.722385 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.722933 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:09.722944 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:09.724832 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:09.726598 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.726606 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.726608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.726845 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.727072 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.727073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.727073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.727190 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.727473 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:09.727480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:09.727627 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 06:36:09.727742 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 06:36:09.727917 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260810 06:36:09.728727 6840 heartbeater.cc:347] Connected to a master server at 127.30.71.62:41073
I20260810 06:36:09.728839 6840 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:09.729024 6840 heartbeater.cc:511] Master 127.30.71.62:41073 requested a full tablet report, sending...
I20260810 06:36:09.729672 6623 ts_manager.cc:194] Registered new tserver with Master: b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533)
I20260810 06:36:09.730523 6623 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:42829
I20260810 06:36:09.732318 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 6701
I20260810 06:36:09.732445 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:36:09.732669 31004 external_mini_cluster.cc:1428] Setting key 50d11ad280a5ba3697fb027a29d20fa2
I20260810 06:36:09.735996 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.729852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:42923 (local address 127.30.71.62:41073)
0810 06:36:09.729914 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:09.729916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:09.730078 (+ 162us) server_negotiation.cc:407] Connection header received
0810 06:36:09.730101 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:09.730102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:09.730112 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:09.730167 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:09.730452 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.731147 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.731881 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.732055 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.732121 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.733174 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:09.733175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:09.733176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:09.733182 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:09.734835 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.735214 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.735215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.735216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.735288 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.735533 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.735534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.735534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.735636 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:09.735691 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:09.735870 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 06:36:09.735936 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:36:09.736107 6848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.729788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41073 (local address 127.30.71.1:42923)
0810 06:36:09.729976 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:09.729985 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:09.730038 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:09.730169 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:09.730170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:09.730326 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:09.730381 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.730383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.731240 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.731241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.731805 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.731808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.732146 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.732147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.732219 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.732667 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:09.732672 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:09.733095 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:09.734921 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.734922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.734923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.735140 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.735371 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.735372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.735373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.735459 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.735704 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:09.735705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:09.735772 (+ 67us) client_negotiation.cc:769] Sending connection context
0810 06:36:09.735855 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:36:09.735974 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab.
Entry for principal oryx/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab.
I20260810 06:36:09.763119 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41073
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab
--principal=oryx/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:09.851438 6851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:09.851665 6851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:09.851688 6851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:09.852783 6851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:09.852818 6851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:09.852837 6851 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:36:09.852854 6851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:09.854281 6851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab
--principal=oryx/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:41073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:09.854472 6851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:09.854645 6851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:09.860008 6857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:09.860016 6856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:09.860060 6860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:09.860172 6851 server_base.cc:1034] running on GCE node
I20260810 06:36:09.860462 6851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:09.860716 6851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:09.861923 6851 hybrid_clock.cc:648] HybridClock initialized: now 1786343769861907 us; error 46 us; skew 500 ppm
Aug 10 06:36:09 dist-test-slave-w206 krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:09.864838 6851 init.cc:377] Logged in from keytab as oryx/127.30.71.2@KRBTEST.COM (short username oryx)
I20260810 06:36:09.865294 6851 webserver.cc:460] Webserver started at http://127.30.71.2:36601/ using document root <none> and password file <none>
I20260810 06:36:09.865525 6851 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:09.865597 6851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:09.865753 6851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:09.866852 6851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "68d1cadd43664f699fa014a56b708857"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "53729a3dc152da1b39d88d86a76b9794"
server_key_iv: "d85d5f2395991d96d08e882cf61d65f6"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.867297 6851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "68d1cadd43664f699fa014a56b708857"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "53729a3dc152da1b39d88d86a76b9794"
server_key_iv: "d85d5f2395991d96d08e882cf61d65f6"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.868659 6851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:09.869475 6867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:09.869664 6851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:09.869791 6851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "68d1cadd43664f699fa014a56b708857"
format_stamp: "Formatted at 2026-08-10 06:36:09 on dist-test-slave-w206"
server_key: "53729a3dc152da1b39d88d86a76b9794"
server_key_iv: "d85d5f2395991d96d08e882cf61d65f6"
server_key_version: "encryptionkey@0"
I20260810 06:36:09.869870 6851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:09.881307 6851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:09.883208 6851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:09.883373 6851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:09.883921 6851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:09.883987 6851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:09.884030 6851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:09.884078 6851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:36:09 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:09.891691 6851 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:41305
I20260810 06:36:09.891746 6987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:41305 every 8 connection(s)
I20260810 06:36:09.892081 6851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:36:09.894451 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.884407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:50415 (local address 127.30.71.62:41073)
0810 06:36:09.884486 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:09.884489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:09.884741 (+ 252us) server_negotiation.cc:407] Connection header received
0810 06:36:09.885548 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:09.885550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:09.885564 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:09.885637 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:09.886591 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.887334 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.888359 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.888621 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.888687 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.891353 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:09.891355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:09.891355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:09.891370 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:09.892995 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.893433 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.893434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.893435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.893502 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.893764 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.893765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.893765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.893891 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:09.893979 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:09.894266 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:36:09.894350 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:36:09.894603 6876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.884376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41073 (local address 127.30.71.2:50415)
0810 06:36:09.884583 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:09.884640 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:09.885392 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:09.885718 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:09.885724 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:09.886145 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:09.886457 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.886464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.887439 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.887440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.888243 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.888245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.888705 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.888706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.888830 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.889398 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:09.889406 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:09.891251 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:09.893101 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.893108 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.893110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.893333 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.893574 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.893575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.893576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.893688 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.893969 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:09.893976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:09.894118 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 06:36:09.894226 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 06:36:09.894385 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260810 06:36:09.895157 6988 heartbeater.cc:347] Connected to a master server at 127.30.71.62:41073
I20260810 06:36:09.895289 6988 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:09.895473 6988 heartbeater.cc:511] Master 127.30.71.62:41073 requested a full tablet report, sending...
I20260810 06:36:09.895898 6623 ts_manager.cc:194] Registered new tserver with Master: 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305)
I20260810 06:36:09.896401 6623 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:50415
I20260810 06:36:09.898916 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 6851
I20260810 06:36:09.899055 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:36:09.899278 31004 external_mini_cluster.cc:1428] Setting key 7958b017eb78f03113f2a7ac8d41bdbe
I20260810 06:36:09.902275 6996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.896190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41073 (local address 127.30.71.2:42155)
0810 06:36:09.896571 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:09.896582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:09.896641 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:09.896774 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:09.896775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:09.896980 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:09.897034 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.897036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.897709 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.897710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.898233 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:09.898235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.898584 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.898585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:09.898633 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.899109 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:09.899113 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:09.899491 (+ 378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:09.901103 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.901104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.901105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.901342 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.901609 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:09.901611 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:09.901611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:09.901684 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:09.902035 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:09.902036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:09.902073 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:36:09.902125 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:36:09.902202 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":63,"threads_started":1}
I20260810 06:36:09.902372 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:09.896253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:42155 (local address 127.30.71.62:41073)
0810 06:36:09.896290 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:09.896292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:09.896632 (+ 340us) server_negotiation.cc:407] Connection header received
0810 06:36:09.896680 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:09.896681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:09.896691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:09.896767 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:09.897133 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.897598 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.898329 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:09.898513 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:09.898570 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:09.899584 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:09.899585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:09.899586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:09.899592 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:09.901012 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.901432 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.901433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.901433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.901505 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:09.901760 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:09.901761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:09.901762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:09.901964 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:09.902043 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:09.902171 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:36:09.902255 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab.
Entry for principal oryx/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab.
I20260810 06:36:09.926518 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41073
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab
--principal=oryx/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:10.010739 6999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:10.011016 6999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:10.011062 6999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:10.012267 6999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:10.012338 6999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:10.012372 6999 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:36:10.012408 6999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:10.013979 6999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab
--principal=oryx/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.71.62:41073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:10.014287 6999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:10.014529 6999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:10.019767 7008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:10.019788 7006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:10.019793 6999 server_base.cc:1034] running on GCE node
W20260810 06:36:10.019767 7005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:10.020301 6999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:10.020561 6999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:10.021724 6999 hybrid_clock.cc:648] HybridClock initialized: now 1786343770021718 us; error 37 us; skew 500 ppm
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343770, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:10.024546 6999 init.cc:377] Logged in from keytab as oryx/127.30.71.3@KRBTEST.COM (short username oryx)
I20260810 06:36:10.024973 6999 webserver.cc:460] Webserver started at http://127.30.71.3:45837/ using document root <none> and password file <none>
I20260810 06:36:10.025161 6999 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:10.025211 6999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:10.025302 6999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:10.026292 6999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "8c36dc6d2e994e1ab6297854721540a0"
format_stamp: "Formatted at 2026-08-10 06:36:10 on dist-test-slave-w206"
server_key: "f97461d08986f0ee05b74d8ffdecd2b0"
server_key_iv: "766e9f4b99d9e04224f4fdfaafe51035"
server_key_version: "encryptionkey@0"
I20260810 06:36:10.026618 6999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "8c36dc6d2e994e1ab6297854721540a0"
format_stamp: "Formatted at 2026-08-10 06:36:10 on dist-test-slave-w206"
server_key: "f97461d08986f0ee05b74d8ffdecd2b0"
server_key_iv: "766e9f4b99d9e04224f4fdfaafe51035"
server_key_version: "encryptionkey@0"
I20260810 06:36:10.027987 6999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:10.028832 7016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.029011 6999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:10.029114 6999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "8c36dc6d2e994e1ab6297854721540a0"
format_stamp: "Formatted at 2026-08-10 06:36:10 on dist-test-slave-w206"
server_key: "f97461d08986f0ee05b74d8ffdecd2b0"
server_key_iv: "766e9f4b99d9e04224f4fdfaafe51035"
server_key_version: "encryptionkey@0"
I20260810 06:36:10.029230 6999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:10.043996 6999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:10.045866 6999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:10.046015 6999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:10.046550 6999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:10.046620 6999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.046661 6999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:10.046689 6999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.054464 6999 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:46591
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343770, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:10.054636 7137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:46591 every 8 connection(s)
I20260810 06:36:10.054906 6999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:36:10.058053 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.047198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:39437 (local address 127.30.71.62:41073)
0810 06:36:10.047286 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.047289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.047525 (+ 236us) server_negotiation.cc:407] Connection header received
0810 06:36:10.048326 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.048328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.048342 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.048435 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:10.049394 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.050218 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.051909 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.052229 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.052323 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.055123 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:10.055124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:10.055125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:10.055134 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:10.056572 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.057038 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.057039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.057040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.057103 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.057373 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.057374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.057374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.057497 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:10.057576 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.057864 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.057963 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:36:10.058254 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.047172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41073 (local address 127.30.71.3:39437)
0810 06:36:10.047375 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.047429 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.048177 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.048533 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.048539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.048975 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:10.049275 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.049281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.050328 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.050329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.051797 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.051800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.052345 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.052346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.052521 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.053197 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:10.053208 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:10.054967 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:10.056686 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:10.056694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:10.056696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:10.056934 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:10.057173 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:10.057174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:10.057174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:10.057284 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:10.057566 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:10.057573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:10.057713 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.057840 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.058002 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260810 06:36:10.058832 7138 heartbeater.cc:347] Connected to a master server at 127.30.71.62:41073
I20260810 06:36:10.058918 7138 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:10.059127 7138 heartbeater.cc:511] Master 127.30.71.62:41073 requested a full tablet report, sending...
I20260810 06:36:10.059626 6623 ts_manager.cc:194] Registered new tserver with Master: 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.060105 6623 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:39437
I20260810 06:36:10.061986 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 6999
I20260810 06:36:10.062116 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:36:10.062306 31004 external_mini_cluster.cc:1428] Setting key d35e4bfaa3acdac42f9d67a5d7c6f89a
I20260810 06:36:10.063370 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:36:10.066309 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.060005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:44041 (local address 127.30.71.62:41073)
0810 06:36:10.060048 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.060050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.060277 (+ 227us) server_negotiation.cc:407] Connection header received
0810 06:36:10.060298 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.060299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.060309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.060362 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:10.060775 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.061352 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.062087 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.062350 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.062429 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.063454 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:10.063455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:10.063456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:10.063463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:10.065016 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.065458 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.065459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.065459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.065554 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.065788 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.065789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.065789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.065901 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:10.065980 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.066139 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.066218 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:36:10.066335 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.059916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41073 (local address 127.30.71.3:44041)
0810 06:36:10.060088 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.060098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.060168 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.060390 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.060391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.060632 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:10.060683 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.060685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.061448 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.061449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.061990 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.061992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.062431 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.062432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.062511 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.062959 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:10.062963 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:10.063367 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:10.065123 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:10.065124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:10.065124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:10.065370 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:10.065647 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:10.065648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:10.065649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:10.065720 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:10.065972 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:10.065973 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:10.066021 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.066113 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.066220 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343770, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:36:10.087731 6678 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:10.082086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50834 (local address 127.30.71.62:41073)
0810 06:36:10.082152 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.082154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.082168 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:36:10.082241 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.082242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.082251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.082311 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:10.082859 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.083426 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.084163 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.084340 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.084388 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.087626 (+ 3238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50834: BlockingRecv error: recv got EOF from 127.0.0.1:50834 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343770, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:10.095347 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.088594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.30.71.62:41073)
0810 06:36:10.088634 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.088635 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.088743 (+ 108us) server_negotiation.cc:407] Connection header received
0810 06:36:10.088779 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.088780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.088788 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.088835 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:10.089264 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.089720 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.090471 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.090650 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.090730 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.092893 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:10.092895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:10.092895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:10.092903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:10.094102 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.094502 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.094503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.094504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.094564 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.094781 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.094782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.094782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.094892 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:10.095043 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.095157 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.095241 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:36:10.097025 6623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:36:10.098628 6623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:10.105844 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.101421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.30.71.2:41305)
0810 06:36:10.101601 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.101606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.101991 (+ 385us) server_negotiation.cc:407] Connection header received
0810 06:36:10.102033 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.102036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.102052 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.102126 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:10.102255 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.103119 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.104763 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.105399 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.105503 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.105582 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.105696 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.105732 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260810 06:36:10.105844 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.101263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41305 (local address 127.0.0.1:57970)
0810 06:36:10.101872 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.101882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.101934 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.102124 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.102124 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.102126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:10.102169 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.102170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.103201 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.103202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.104659 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.104661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.105506 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.105507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.105592 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.105606 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.105681 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.105728 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":65,"thread_start_us":83,"threads_started":1}
I20260810 06:36:10.106045 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.101200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.30.71.3:46591)
0810 06:36:10.101642 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.101646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.101659 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:36:10.101699 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.101701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.101719 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.102357 (+ 638us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:10.102413 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.103479 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.104652 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.105441 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.105515 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.105589 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.105886 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.105926 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":93,"threads_started":1}
I20260810 06:36:10.105865 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.101055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:39533 (local address 127.0.0.1:34364)
0810 06:36:10.101178 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.101219 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.101304 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.101894 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.101899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.101902 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:10.101981 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.101983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.103462 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.103463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.104658 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.104661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.105546 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.105547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.105617 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.105621 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.105659 (+ 38us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.105728 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":11,"thread_start_us":72,"threads_started":1}
I20260810 06:36:10.106312 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.101496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.30.71.1:39533)
0810 06:36:10.101697 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.101701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.101710 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:10.101747 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.101749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.101763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.102060 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:10.102120 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.103368 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.104741 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.105440 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.105893 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.105978 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.106005 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.106214 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260810 06:36:10.106462 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.101055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46591 (local address 127.0.0.1:43528)
0810 06:36:10.101474 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.101482 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.101552 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.102257 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.102258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.102260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:10.102295 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.102296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.103560 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.103561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.104517 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.104520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.105618 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.105618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.105675 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.105677 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.105722 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.105752 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":4,"spinlock_wait_cycles":384,"thread_start_us":80,"threads_started":1}
I20260810 06:36:10.106958 6771 tablet_service.cc:1467] Processing CreateTablet for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 (DEFAULT_TABLE table=test-table [id=2fa7149e405e4185ae737e3f9cf62cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.107036 6918 tablet_service.cc:1467] Processing CreateTablet for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 (DEFAULT_TABLE table=test-table [id=2fa7149e405e4185ae737e3f9cf62cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.107123 7066 tablet_service.cc:1467] Processing CreateTablet for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 (DEFAULT_TABLE table=test-table [id=2fa7149e405e4185ae737e3f9cf62cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.107420 7066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.107420 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.107424 6771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.110555 7167 tablet_bootstrap.cc:492] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Bootstrap starting.
I20260810 06:36:10.111152 7168 tablet_bootstrap.cc:492] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Bootstrap starting.
I20260810 06:36:10.111265 7167 tablet_bootstrap.cc:654] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.111565 7170 tablet_bootstrap.cc:492] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Bootstrap starting.
I20260810 06:36:10.111629 7167 log.cc:824] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:10.111789 7168 tablet_bootstrap.cc:654] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.112134 7168 log.cc:824] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:10.112224 7170 tablet_bootstrap.cc:654] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.112497 7167 tablet_bootstrap.cc:492] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: No bootstrap required, opened a new log
I20260810 06:36:10.112538 7170 log.cc:824] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:10.112562 7167 ts_tablet_manager.cc:1397] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:10.112844 7168 tablet_bootstrap.cc:492] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: No bootstrap required, opened a new log
I20260810 06:36:10.112906 7168 ts_tablet_manager.cc:1397] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:10.113339 7170 tablet_bootstrap.cc:492] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: No bootstrap required, opened a new log
I20260810 06:36:10.113396 7170 ts_tablet_manager.cc:1397] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:10.114481 7168 raft_consensus.cc:348] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.114607 7168 raft_consensus.cc:374] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.114629 7168 raft_consensus.cc:729] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.114754 7168 consensus_queue.cc:260] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.114671 7167 raft_consensus.cc:348] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.114835 7167 raft_consensus.cc:374] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.114868 7167 raft_consensus.cc:729] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26bcda9d15344f9a6f472d1e0edb153, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.115056 6988 heartbeater.cc:503] Master 127.30.71.62:41073 was elected leader, sending a full tablet report...
I20260810 06:36:10.115347 7168 ts_tablet_manager.cc:1428] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:36:10.115360 7167 consensus_queue.cc:260] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.115458 7170 raft_consensus.cc:348] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.115605 7170 raft_consensus.cc:374] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.115639 7170 raft_consensus.cc:729] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c36dc6d2e994e1ab6297854721540a0, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.115670 7167 ts_tablet_manager.cc:1428] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:36:10.115748 7170 consensus_queue.cc:260] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.115805 6840 heartbeater.cc:503] Master 127.30.71.62:41073 was elected leader, sending a full tablet report...
I20260810 06:36:10.116025 7170 ts_tablet_manager.cc:1428] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:36:10.116155 7138 heartbeater.cc:503] Master 127.30.71.62:41073 was elected leader, sending a full tablet report...
W20260810 06:36:10.142690 6989 tablet.cc:2369] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:36:10.225612 6841 tablet.cc:2369] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:36:10.305572 7139 tablet.cc:2369] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:36:10.359580 7178 raft_consensus.cc:482] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:10.359758 7178 raft_consensus.cc:504] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.360157 7178 leader_election.cc:288] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305)
I20260810 06:36:10.363683 7176 raft_consensus.cc:482] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:10.363847 7176 raft_consensus.cc:504] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.364231 7176 leader_election.cc:288] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.364410 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.360591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41305 (local address 127.30.71.3:42733)
0810 06:36:10.360767 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.360778 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.360844 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.361033 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.361034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.361035 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:10.361097 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.361098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.361934 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.361935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.363141 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.363145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.364089 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.364091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.364187 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.364190 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.364236 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.364278 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":5,"thread_start_us":98,"threads_started":1}
I20260810 06:36:10.364621 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.360692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:42733 (local address 127.30.71.2:41305)
0810 06:36:10.360787 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.360790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.360836 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:36:10.360938 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.360939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.360952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.361010 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:10.361173 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.361844 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.363374 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.363958 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.364037 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.364047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.364514 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.364566 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:36:10.364719 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.360495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:56163 (local address 127.30.71.1:39533)
0810 06:36:10.360567 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.360569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.360581 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:36:10.360669 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.360671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.360685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.360740 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:10.360927 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.361721 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.363374 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.363958 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.364037 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.364047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.364584 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.364630 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:36:10.364904 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.360410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:39533 (local address 127.30.71.3:56163)
0810 06:36:10.360495 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.360506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.360594 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.360764 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.360765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.360767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:10.360843 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.360846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.361813 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.361814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.363141 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.363145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.364089 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.364091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.364174 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.364177 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.364230 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.364279 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":12}
I20260810 06:36:10.365056 6940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b4d27bbf6d4683a79f7c31e4df7fc3" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68d1cadd43664f699fa014a56b708857" is_pre_election: true
I20260810 06:36:10.365161 6940 raft_consensus.cc:2454] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 0.
I20260810 06:36:10.365182 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b4d27bbf6d4683a79f7c31e4df7fc3" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153" is_pre_election: true
I20260810 06:36:10.365317 6791 raft_consensus.cc:2454] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 0.
I20260810 06:36:10.365368 7017 leader_election.cc:302] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, 8c36dc6d2e994e1ab6297854721540a0; no voters:
I20260810 06:36:10.365583 7178 raft_consensus.cc:2781] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:10.365687 7178 raft_consensus.cc:482] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:10.365737 7178 raft_consensus.cc:3037] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.367100 7178 raft_consensus.cc:504] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.367422 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b4d27bbf6d4683a79f7c31e4df7fc3" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153"
I20260810 06:36:10.367470 6940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b4d27bbf6d4683a79f7c31e4df7fc3" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68d1cadd43664f699fa014a56b708857"
I20260810 06:36:10.367530 6940 raft_consensus.cc:3037] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.367514 6791 raft_consensus.cc:3037] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.367974 6876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.364412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:39533 (local address 127.30.71.2:41275)
0810 06:36:10.364470 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.364482 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.364524 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.364870 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.364871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.364873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:10.364930 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.364932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.365786 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.365787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.366998 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.367001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.367746 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.367747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.367837 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.367841 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.367870 (+ 29us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.367909 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:36:10.368220 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.364534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:41275 (local address 127.30.71.1:39533)
0810 06:36:10.364756 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.364758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.364766 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:10.364788 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.364789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.364801 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.364865 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:10.365024 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.365715 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.367082 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.367635 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.368070 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.368085 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.368104 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.368146 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260810 06:36:10.368396 6940 raft_consensus.cc:2454] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 1.
I20260810 06:36:10.368397 6791 raft_consensus.cc:2454] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 1.
I20260810 06:36:10.368592 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.365143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:46905 (local address 127.30.71.3:46591)
0810 06:36:10.365185 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.365188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.365365 (+ 177us) server_negotiation.cc:407] Connection header received
0810 06:36:10.365474 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.365475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.365485 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.365515 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:10.365752 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.366193 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.367745 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.368277 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.368334 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.368345 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.368503 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.368535 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:36:10.368607 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.364431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46591 (local address 127.30.71.2:46905)
0810 06:36:10.365311 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.365322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.365389 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.365577 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.365577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.365579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:10.365649 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.365651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.366287 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.366288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.367647 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.367650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.368357 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.368358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.368421 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.368423 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.368498 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.368535 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":104,"threads_started":1}
I20260810 06:36:10.368631 7178 leader_election.cc:288] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 election: Requested vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305)
I20260810 06:36:10.368666 7017 leader_election.cc:302] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, 8c36dc6d2e994e1ab6297854721540a0; no voters:
I20260810 06:36:10.368728 7178 raft_consensus.cc:2781] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:10.368795 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b4d27bbf6d4683a79f7c31e4df7fc3" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153" is_pre_election: true
I20260810 06:36:10.368873 7178 raft_consensus.cc:686] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c36dc6d2e994e1ab6297854721540a0, State: Running, Role: LEADER
I20260810 06:36:10.368889 6791 raft_consensus.cc:2379] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68d1cadd43664f699fa014a56b708857 in current term 1: Already voted for candidate 8c36dc6d2e994e1ab6297854721540a0 in this term.
I20260810 06:36:10.368956 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b4d27bbf6d4683a79f7c31e4df7fc3" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c36dc6d2e994e1ab6297854721540a0" is_pre_election: true
I20260810 06:36:10.368961 7178 consensus_queue.cc:237] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.369563 6869 leader_election.cc:302] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68d1cadd43664f699fa014a56b708857; no voters: 8c36dc6d2e994e1ab6297854721540a0, b26bcda9d15344f9a6f472d1e0edb153
I20260810 06:36:10.369727 7176 raft_consensus.cc:2726] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 06:36:10.369979 6624 catalog_manager.cc:5495] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3). New cstate: current_term: 1 leader_uuid: "8c36dc6d2e994e1ab6297854721540a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } health_report { overall_health: HEALTHY } } }
I20260810 06:36:10.389971 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.387276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43532 (local address 127.30.71.3:46591)
0810 06:36:10.387333 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.387336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.387344 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:10.387463 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.387464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.387490 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.387571 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:10.387725 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.388332 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.389160 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.389370 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.389449 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.389628 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:10.389738 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:10.389836 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.389862 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.389883 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:36:10.392104 6940 raft_consensus.cc:1261] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Refusing update from remote peer 8c36dc6d2e994e1ab6297854721540a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.392086 6791 raft_consensus.cc:1261] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Refusing update from remote peer 8c36dc6d2e994e1ab6297854721540a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.392446 7178 consensus_queue.cc:1035] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.392608 7182 consensus_queue.cc:1035] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.396297 7187 mvcc.cc:205] Tried to move back new op lower bound from 7316864083522969600 to 7316864083432706048. Current Snapshot: MvccSnapshot[applied={T|T < 7316864083522969600}]
I20260810 06:36:10.397557 6623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:10.397646 6623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:10.398453 6623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Sending DeleteTablet for 3 replicas of tablet 79b4d27bbf6d4683a79f7c31e4df7fc3
I20260810 06:36:10.398837 6918 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:57970
I20260810 06:36:10.398818 7066 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:43528
I20260810 06:36:10.398857 6771 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:34364
I20260810 06:36:10.399116 7197 tablet_replica.cc:330] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: stopping tablet replica
I20260810 06:36:10.399223 7196 tablet_replica.cc:330] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: stopping tablet replica
I20260810 06:36:10.399278 7197 raft_consensus.cc:2229] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:10.399348 7196 raft_consensus.cc:2229] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:36:10.399439 7197 raft_consensus.cc:2258] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.399500 7196 raft_consensus.cc:2258] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.399441 6623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:36:10.399593 6623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:10.399966 7198 tablet_replica.cc:330] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: stopping tablet replica
I20260810 06:36:10.400200 7196 ts_tablet_manager.cc:1905] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.400341 7197 ts_tablet_manager.cc:1905] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.401582 6918 tablet_service.cc:1467] Processing CreateTablet for tablet cbe9de936b5a4337b5f5da3b66b2de9a (DEFAULT_TABLE table=test-table [id=31ed89c9432842bba9386b9f300d8eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.401705 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbe9de936b5a4337b5f5da3b66b2de9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.402259 7196 ts_tablet_manager.cc:1918] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:10.402248 7197 ts_tablet_manager.cc:1918] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:10.402329 7196 log.cc:1196] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/79b4d27bbf6d4683a79f7c31e4df7fc3
I20260810 06:36:10.402374 7197 log.cc:1196] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/79b4d27bbf6d4683a79f7c31e4df7fc3
I20260810 06:36:10.402614 7196 ts_tablet_manager.cc:1939] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 8c36dc6d2e994e1ab6297854721540a0: Deleting consensus metadata
I20260810 06:36:10.402613 7197 ts_tablet_manager.cc:1939] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P 68d1cadd43664f699fa014a56b708857: Deleting consensus metadata
I20260810 06:36:10.403028 6606 catalog_manager.cc:4841] TS 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305): tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 (table test-table [id=2fa7149e405e4185ae737e3f9cf62cdf]) successfully deleted
I20260810 06:36:10.403028 6607 catalog_manager.cc:4841] TS 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591): tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 (table test-table [id=2fa7149e405e4185ae737e3f9cf62cdf]) successfully deleted
I20260810 06:36:10.403101 7168 tablet_bootstrap.cc:492] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: Bootstrap starting.
I20260810 06:36:10.403169 7066 tablet_service.cc:1467] Processing CreateTablet for tablet cbe9de936b5a4337b5f5da3b66b2de9a (DEFAULT_TABLE table=test-table [id=31ed89c9432842bba9386b9f300d8eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.403271 7066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbe9de936b5a4337b5f5da3b66b2de9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.403507 7168 tablet_bootstrap.cc:654] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.404166 7168 tablet_bootstrap.cc:492] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: No bootstrap required, opened a new log
I20260810 06:36:10.404234 7168 ts_tablet_manager.cc:1397] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:10.404381 7168 raft_consensus.cc:348] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.404431 7168 raft_consensus.cc:374] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.404448 7168 raft_consensus.cc:729] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.404505 7168 consensus_queue.cc:260] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.404595 7168 ts_tablet_manager.cc:1428] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.404707 7170 tablet_bootstrap.cc:492] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: Bootstrap starting.
I20260810 06:36:10.405128 7170 tablet_bootstrap.cc:654] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.405774 7170 tablet_bootstrap.cc:492] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: No bootstrap required, opened a new log
I20260810 06:36:10.405817 7170 ts_tablet_manager.cc:1397] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:10.405956 7170 raft_consensus.cc:348] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.406013 7170 raft_consensus.cc:374] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.406032 7170 raft_consensus.cc:729] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c36dc6d2e994e1ab6297854721540a0, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.406085 7170 consensus_queue.cc:260] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.406198 7170 ts_tablet_manager.cc:1428] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.405822 6771 tablet_service.cc:1467] Processing CreateTablet for tablet cbe9de936b5a4337b5f5da3b66b2de9a (DEFAULT_TABLE table=test-table [id=31ed89c9432842bba9386b9f300d8eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.407073 6771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbe9de936b5a4337b5f5da3b66b2de9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.408469 7167 tablet_bootstrap.cc:492] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: Bootstrap starting.
I20260810 06:36:10.407054 7198 raft_consensus.cc:2229] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:10.408928 7198 raft_consensus.cc:2258] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.408946 7167 tablet_bootstrap.cc:654] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.409524 7198 ts_tablet_manager.cc:1905] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.409636 7167 tablet_bootstrap.cc:492] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: No bootstrap required, opened a new log
I20260810 06:36:10.409686 7167 ts_tablet_manager.cc:1397] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:10.409847 7167 raft_consensus.cc:348] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.409895 7167 raft_consensus.cc:374] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.409909 7167 raft_consensus.cc:729] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26bcda9d15344f9a6f472d1e0edb153, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.409971 7167 consensus_queue.cc:260] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.410080 7167 ts_tablet_manager.cc:1428] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.411765 7198 ts_tablet_manager.cc:1918] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:10.411846 7198 log.cc:1196] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/79b4d27bbf6d4683a79f7c31e4df7fc3
I20260810 06:36:10.412132 7198 ts_tablet_manager.cc:1939] T 79b4d27bbf6d4683a79f7c31e4df7fc3 P b26bcda9d15344f9a6f472d1e0edb153: Deleting consensus metadata
I20260810 06:36:10.412567 6606 catalog_manager.cc:4841] TS b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533): tablet 79b4d27bbf6d4683a79f7c31e4df7fc3 (table test-table [id=2fa7149e405e4185ae737e3f9cf62cdf]) successfully deleted
I20260810 06:36:10.598111 7176 raft_consensus.cc:482] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:10.598232 7176 raft_consensus.cc:504] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.598446 7176 leader_election.cc:288] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.598713 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbe9de936b5a4337b5f5da3b66b2de9a" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153" is_pre_election: true
I20260810 06:36:10.598726 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbe9de936b5a4337b5f5da3b66b2de9a" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c36dc6d2e994e1ab6297854721540a0" is_pre_election: true
I20260810 06:36:10.598824 6791 raft_consensus.cc:2454] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 0.
I20260810 06:36:10.598878 7086 raft_consensus.cc:2454] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 0.
I20260810 06:36:10.599078 6868 leader_election.cc:302] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, b26bcda9d15344f9a6f472d1e0edb153; no voters:
I20260810 06:36:10.599213 7176 raft_consensus.cc:2781] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:10.599255 7176 raft_consensus.cc:482] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:10.599277 7176 raft_consensus.cc:3037] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.600005 7176 raft_consensus.cc:504] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.600162 7176 leader_election.cc:288] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 election: Requested vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.600368 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbe9de936b5a4337b5f5da3b66b2de9a" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153"
I20260810 06:36:10.600361 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbe9de936b5a4337b5f5da3b66b2de9a" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c36dc6d2e994e1ab6297854721540a0"
I20260810 06:36:10.600441 6791 raft_consensus.cc:3037] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.600461 7086 raft_consensus.cc:3037] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.601154 6791 raft_consensus.cc:2454] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 1.
I20260810 06:36:10.601310 6868 leader_election.cc:302] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, b26bcda9d15344f9a6f472d1e0edb153; no voters:
I20260810 06:36:10.601326 7086 raft_consensus.cc:2454] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 1.
I20260810 06:36:10.601424 7176 raft_consensus.cc:2781] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:10.601544 7176 raft_consensus.cc:686] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 1 LEADER]: Becoming Leader. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Running, Role: LEADER
I20260810 06:36:10.601634 7176 consensus_queue.cc:237] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.602415 6622 catalog_manager.cc:5495] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68d1cadd43664f699fa014a56b708857 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343769, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.62@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:10.663069 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.655838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.30.71.62:41073)
0810 06:36:10.656021 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.656023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.656057 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:36:10.656135 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.656135 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.656146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.656214 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:10.656732 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.657278 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.657960 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.658149 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.658198 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.660476 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:10.660477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:10.660477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:10.660484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:10.661974 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.662319 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.662320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.662320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.662378 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.662567 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.662567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.662568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.662678 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:10.662730 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.662850 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.662926 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260810 06:36:10.663195 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.655790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41073 (local address 127.0.0.1:50852)
0810 06:36:10.655984 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.656000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.656075 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.656233 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.656234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.656584 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:10.656638 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.656640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.657358 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.657359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.657886 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.657887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.658215 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.658216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.658263 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.658776 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:10.658823 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:10.660392 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:10.662048 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:10.662055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:10.662057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:10.662247 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:10.662450 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:10.662451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:10.662451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:10.662513 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:10.662733 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:10.662739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:10.662783 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.662835 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.663055 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":68,"thread_start_us":99,"threads_started":1}
I20260810 06:36:10.665222 6622 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:50852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:36:10.665464 6622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:10.666967 6918 tablet_service.cc:1467] Processing CreateTablet for tablet b00ac459cfaa46f485a4145bec0b3112 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17eadc67d29f4bbc8f9832842733db0d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:36:10.666996 6771 tablet_service.cc:1467] Processing CreateTablet for tablet b00ac459cfaa46f485a4145bec0b3112 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17eadc67d29f4bbc8f9832842733db0d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:36:10.667167 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00ac459cfaa46f485a4145bec0b3112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.667173 6771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00ac459cfaa46f485a4145bec0b3112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.668707 7167 tablet_bootstrap.cc:492] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: Bootstrap starting.
I20260810 06:36:10.668706 7168 tablet_bootstrap.cc:492] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: Bootstrap starting.
I20260810 06:36:10.668785 7066 tablet_service.cc:1467] Processing CreateTablet for tablet b00ac459cfaa46f485a4145bec0b3112 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17eadc67d29f4bbc8f9832842733db0d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:36:10.668937 7066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00ac459cfaa46f485a4145bec0b3112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.669200 7168 tablet_bootstrap.cc:654] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.669200 7167 tablet_bootstrap.cc:654] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.669894 7167 tablet_bootstrap.cc:492] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: No bootstrap required, opened a new log
I20260810 06:36:10.669966 7167 ts_tablet_manager.cc:1397] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:10.670127 7168 tablet_bootstrap.cc:492] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: No bootstrap required, opened a new log
I20260810 06:36:10.670164 7168 ts_tablet_manager.cc:1397] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:10.670122 7167 raft_consensus.cc:348] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.670200 7167 raft_consensus.cc:374] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.670228 7167 raft_consensus.cc:729] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26bcda9d15344f9a6f472d1e0edb153, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.670218 7170 tablet_bootstrap.cc:492] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: Bootstrap starting.
I20260810 06:36:10.670288 7167 consensus_queue.cc:260] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.670300 7168 raft_consensus.cc:348] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.670361 7168 raft_consensus.cc:374] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.670380 7168 raft_consensus.cc:729] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.670384 7167 ts_tablet_manager.cc:1428] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.670425 7168 consensus_queue.cc:260] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.670527 7168 ts_tablet_manager.cc:1428] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.670631 7170 tablet_bootstrap.cc:654] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.670584 7176 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.670679 7176 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:36:10.670650 7177 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.670734 7177 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:36:10.671422 7170 tablet_bootstrap.cc:492] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: No bootstrap required, opened a new log
I20260810 06:36:10.671476 7170 ts_tablet_manager.cc:1397] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:10.671561 7170 raft_consensus.cc:348] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.671612 7170 raft_consensus.cc:374] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.671625 7170 raft_consensus.cc:729] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c36dc6d2e994e1ab6297854721540a0, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.671691 7170 consensus_queue.cc:260] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.671797 7170 ts_tablet_manager.cc:1428] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.671826 7192 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.671913 7192 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:36:10.710166 7176 raft_consensus.cc:482] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:10.710266 7176 raft_consensus.cc:504] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.710433 7176 leader_election.cc:288] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.710672 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00ac459cfaa46f485a4145bec0b3112" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153" is_pre_election: true
I20260810 06:36:10.710669 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00ac459cfaa46f485a4145bec0b3112" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c36dc6d2e994e1ab6297854721540a0" is_pre_election: true
I20260810 06:36:10.710773 6791 raft_consensus.cc:2454] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 0.
I20260810 06:36:10.710799 7086 raft_consensus.cc:2454] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 0.
I20260810 06:36:10.710955 6868 leader_election.cc:302] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, b26bcda9d15344f9a6f472d1e0edb153; no voters:
I20260810 06:36:10.711120 7176 raft_consensus.cc:2781] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:10.711165 7176 raft_consensus.cc:482] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:10.711181 7176 raft_consensus.cc:3037] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.711864 7176 raft_consensus.cc:504] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.712038 7176 leader_election.cc:288] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 election: Requested vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.712234 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00ac459cfaa46f485a4145bec0b3112" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c36dc6d2e994e1ab6297854721540a0"
I20260810 06:36:10.712323 7086 raft_consensus.cc:3037] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.712219 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00ac459cfaa46f485a4145bec0b3112" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153"
I20260810 06:36:10.712363 6791 raft_consensus.cc:3037] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.712968 6791 raft_consensus.cc:2454] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 1.
I20260810 06:36:10.713022 7086 raft_consensus.cc:2454] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 1.
I20260810 06:36:10.713194 6869 leader_election.cc:302] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, 8c36dc6d2e994e1ab6297854721540a0; no voters:
I20260810 06:36:10.713310 7176 raft_consensus.cc:2781] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:10.713403 7176 raft_consensus.cc:686] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [term 1 LEADER]: Becoming Leader. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Running, Role: LEADER
I20260810 06:36:10.713485 7176 consensus_queue.cc:237] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.713896 7203 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: TxnStatusTablet state changed. Reason: New leader 68d1cadd43664f699fa014a56b708857. Latest consensus state: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.713994 7203 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:36:10.714210 7218 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:36:10.714244 6623 catalog_manager.cc:5495] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68d1cadd43664f699fa014a56b708857 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } health_report { overall_health: UNKNOWN } } }
I20260810 06:36:10.714341 7218 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:36:10.731421 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.728612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41305 (local address 127.0.0.1:57974)
0810 06:36:10.728721 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.728734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.728805 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.728999 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.729000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.729002 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:36:10.729056 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.729059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.729737 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.729739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.730388 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.730390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.730797 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.730798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.730859 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.730876 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:36:10.731203 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:36:10.731214 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.731260 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.731317 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:36:10.731387 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.728669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.30.71.2:41305)
0810 06:36:10.728723 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.728726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.728769 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:36:10.728894 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.728895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.728920 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.728993 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:10.729118 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.729654 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.730482 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.730703 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.730793 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.731011 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:10.731120 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:10.731202 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.731281 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.731304 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:36:10.732656 6791 raft_consensus.cc:1261] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Refusing update from remote peer 68d1cadd43664f699fa014a56b708857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.732745 7086 raft_consensus.cc:1261] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Refusing update from remote peer 68d1cadd43664f699fa014a56b708857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.732899 7203 consensus_queue.cc:1035] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.733048 7176 consensus_queue.cc:1035] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.733526 7177 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: TxnStatusTablet state changed. Reason: New leader 68d1cadd43664f699fa014a56b708857. Latest consensus state: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.733631 7177 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:36:10.733760 7192 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: TxnStatusTablet state changed. Reason: New leader 68d1cadd43664f699fa014a56b708857. Latest consensus state: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.733835 7192 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:36:10.734406 7176 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.734521 7176 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:36:10.734932 7203 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.735044 7203 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P 68d1cadd43664f699fa014a56b708857: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:36:10.735180 7185 mvcc.cc:205] Tried to move back new op lower bound from 7316864084918837248 to 7316864084842774528. Current Snapshot: MvccSnapshot[applied={T|T < 7316864084918837248}]
I20260810 06:36:10.735332 7192 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.735399 7192 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P 8c36dc6d2e994e1ab6297854721540a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:36:10.735409 7177 tablet_replica.cc:439] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } } }
I20260810 06:36:10.735497 7177 tablet_replica.cc:442] T b00ac459cfaa46f485a4145bec0b3112 P b26bcda9d15344f9a6f472d1e0edb153: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:36:10.739336 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.736990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.30.71.2:41305)
0810 06:36:10.737029 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.737031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.737040 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:10.737076 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.737077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.737089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.737135 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:10.737240 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.737709 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.738615 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.738824 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.738912 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.739084 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:10.739115 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:10.739183 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.739243 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.739260 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:36:10.744566 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.741954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:58893 (local address 127.30.71.2:41305)
0810 06:36:10.742015 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.742018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.742130 (+ 112us) server_negotiation.cc:407] Connection header received
0810 06:36:10.742194 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.742195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.742203 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.742243 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:10.742364 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.742776 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.743915 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.744106 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.744164 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.744357 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:10.744380 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:10.744420 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.744475 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.744497 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":36}
I20260810 06:36:10.745172 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.741894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41305 (local address 127.30.71.2:58893)
0810 06:36:10.742065 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:10.742076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:10.742133 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:10.742243 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:10.742244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:10.742245 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:36:10.742309 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.742310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.742850 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.742851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.743817 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:10.743819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.744176 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.744177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:10.744261 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.744276 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:36:10.744422 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:36:10.744423 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 06:36:10.744474 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:36:10.744516 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260810 06:36:10.747478 7086 raft_consensus.cc:1261] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Refusing update from remote peer 68d1cadd43664f699fa014a56b708857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.747488 6791 raft_consensus.cc:1261] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Refusing update from remote peer 68d1cadd43664f699fa014a56b708857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.747694 7226 consensus_queue.cc:1035] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.747820 7176 consensus_queue.cc:1035] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.749590 7186 mvcc.cc:205] Tried to move back new op lower bound from 7316864084980170752 to 7316864084384948224. Current Snapshot: MvccSnapshot[applied={T|T < 7316864084980170752}]
I20260810 06:36:10.759550 6622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:10.759655 6622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:10.760311 6622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Sending DeleteTablet for 3 replicas of tablet cbe9de936b5a4337b5f5da3b66b2de9a
I20260810 06:36:10.760670 7066 tablet_service.cc:1514] Processing DeleteTablet for tablet cbe9de936b5a4337b5f5da3b66b2de9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:43528
I20260810 06:36:10.760675 6771 tablet_service.cc:1514] Processing DeleteTablet for tablet cbe9de936b5a4337b5f5da3b66b2de9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:34364
I20260810 06:36:10.760771 7196 tablet_replica.cc:330] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: stopping tablet replica
I20260810 06:36:10.760775 7198 tablet_replica.cc:330] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: stopping tablet replica
I20260810 06:36:10.760820 7196 raft_consensus.cc:2229] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:10.760823 7198 raft_consensus.cc:2229] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:10.760855 7196 raft_consensus.cc:2258] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.760857 7198 raft_consensus.cc:2258] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.761166 7198 ts_tablet_manager.cc:1905] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.761237 6918 tablet_service.cc:1514] Processing DeleteTablet for tablet cbe9de936b5a4337b5f5da3b66b2de9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:57970
I20260810 06:36:10.761341 7197 tablet_replica.cc:330] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: stopping tablet replica
I20260810 06:36:10.761389 7197 raft_consensus.cc:2229] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:36:10.761458 7197 raft_consensus.cc:2258] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.763154 7198 ts_tablet_manager.cc:1918] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:36:10.763218 7198 log.cc:1196] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/cbe9de936b5a4337b5f5da3b66b2de9a
I20260810 06:36:10.763451 7198 ts_tablet_manager.cc:1939] T cbe9de936b5a4337b5f5da3b66b2de9a P b26bcda9d15344f9a6f472d1e0edb153: Deleting consensus metadata
I20260810 06:36:10.763636 7197 ts_tablet_manager.cc:1905] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.763695 7196 ts_tablet_manager.cc:1905] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.764029 6606 catalog_manager.cc:4841] TS b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533): tablet cbe9de936b5a4337b5f5da3b66b2de9a (table test-table [id=31ed89c9432842bba9386b9f300d8eb5]) successfully deleted
I20260810 06:36:10.765321 7196 ts_tablet_manager.cc:1918] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:36:10.765395 7196 log.cc:1196] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/cbe9de936b5a4337b5f5da3b66b2de9a
I20260810 06:36:10.765388 7197 ts_tablet_manager.cc:1918] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:36:10.765451 7197 log.cc:1196] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/cbe9de936b5a4337b5f5da3b66b2de9a
I20260810 06:36:10.765625 7196 ts_tablet_manager.cc:1939] T cbe9de936b5a4337b5f5da3b66b2de9a P 8c36dc6d2e994e1ab6297854721540a0: Deleting consensus metadata
I20260810 06:36:10.765658 7197 ts_tablet_manager.cc:1939] T cbe9de936b5a4337b5f5da3b66b2de9a P 68d1cadd43664f699fa014a56b708857: Deleting consensus metadata
I20260810 06:36:10.766050 6606 catalog_manager.cc:4841] TS 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305): tablet cbe9de936b5a4337b5f5da3b66b2de9a (table test-table [id=31ed89c9432842bba9386b9f300d8eb5]) successfully deleted
I20260810 06:36:10.766175 6607 catalog_manager.cc:4841] TS 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591): tablet cbe9de936b5a4337b5f5da3b66b2de9a (table test-table [id=31ed89c9432842bba9386b9f300d8eb5]) successfully deleted
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:36:10.788437 7212 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:10.782659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.30.71.62:41073)
0810 06:36:10.782727 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.782729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.782860 (+ 131us) server_negotiation.cc:407] Connection header received
0810 06:36:10.783005 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.783006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.783020 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.783094 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:10.783684 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.784200 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.784983 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.785198 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.785286 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.788351 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50858: BlockingRecv error: recv got EOF from 127.0.0.1:50858 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 06:36:10 dist-test-slave-w206 krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM
I20260810 06:36:10.797243 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.789302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.30.71.62:41073)
0810 06:36:10.789345 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.789347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.789417 (+ 70us) server_negotiation.cc:407] Connection header received
0810 06:36:10.789515 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.789516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.789526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.789573 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:10.790160 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.790616 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.791668 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.791849 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.791910 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.794319 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:10.794320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:10.794321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:10.794329 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:10.796078 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.796430 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.796431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.796431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.796499 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:10.796724 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:10.796725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:10.796726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:10.796870 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:10.796989 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.797063 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.797149 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:36:10.798892 6622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:36:10.799110 6622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:10.800608 6771 tablet_service.cc:1467] Processing CreateTablet for tablet 8b845ad7e04f401fa7f85751f8c71a04 (DEFAULT_TABLE table=test-table [id=ba6423b294134baaa8eb341c710d05eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.800626 7066 tablet_service.cc:1467] Processing CreateTablet for tablet 8b845ad7e04f401fa7f85751f8c71a04 (DEFAULT_TABLE table=test-table [id=ba6423b294134baaa8eb341c710d05eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.800777 6771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b845ad7e04f401fa7f85751f8c71a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.800784 7066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b845ad7e04f401fa7f85751f8c71a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.802073 7167 tablet_bootstrap.cc:492] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: Bootstrap starting.
I20260810 06:36:10.802206 6918 tablet_service.cc:1467] Processing CreateTablet for tablet 8b845ad7e04f401fa7f85751f8c71a04 (DEFAULT_TABLE table=test-table [id=ba6423b294134baaa8eb341c710d05eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.802349 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b845ad7e04f401fa7f85751f8c71a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.802575 7167 tablet_bootstrap.cc:654] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.803296 7167 tablet_bootstrap.cc:492] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: No bootstrap required, opened a new log
I20260810 06:36:10.803367 7167 ts_tablet_manager.cc:1397] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:10.803544 7167 raft_consensus.cc:348] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.803630 7167 raft_consensus.cc:374] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.803659 7167 raft_consensus.cc:729] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26bcda9d15344f9a6f472d1e0edb153, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.803696 7170 tablet_bootstrap.cc:492] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: Bootstrap starting.
I20260810 06:36:10.803701 7167 consensus_queue.cc:260] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.803797 7167 ts_tablet_manager.cc:1428] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:10.804046 7170 tablet_bootstrap.cc:654] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.804332 7168 tablet_bootstrap.cc:492] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: Bootstrap starting.
I20260810 06:36:10.804761 7168 tablet_bootstrap.cc:654] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.805557 7170 tablet_bootstrap.cc:492] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: No bootstrap required, opened a new log
I20260810 06:36:10.805616 7170 ts_tablet_manager.cc:1397] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:10.805716 7170 raft_consensus.cc:348] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.805787 7170 raft_consensus.cc:374] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.805809 7170 raft_consensus.cc:729] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c36dc6d2e994e1ab6297854721540a0, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.805851 7170 consensus_queue.cc:260] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.805940 7170 ts_tablet_manager.cc:1428] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.806267 7168 tablet_bootstrap.cc:492] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: No bootstrap required, opened a new log
I20260810 06:36:10.806320 7168 ts_tablet_manager.cc:1397] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:10.806470 7168 raft_consensus.cc:348] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.806545 7168 raft_consensus.cc:374] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.806574 7168 raft_consensus.cc:729] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.806617 7168 consensus_queue.cc:260] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.806704 7168 ts_tablet_manager.cc:1428] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.938208 7226 raft_consensus.cc:482] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:10.938316 7226 raft_consensus.cc:504] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.938550 7226 leader_election.cc:288] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.938817 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b845ad7e04f401fa7f85751f8c71a04" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c36dc6d2e994e1ab6297854721540a0" is_pre_election: true
I20260810 06:36:10.938851 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b845ad7e04f401fa7f85751f8c71a04" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153" is_pre_election: true
I20260810 06:36:10.938946 7086 raft_consensus.cc:2454] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 0.
I20260810 06:36:10.938963 6791 raft_consensus.cc:2454] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 0.
I20260810 06:36:10.939191 6868 leader_election.cc:302] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, b26bcda9d15344f9a6f472d1e0edb153; no voters:
I20260810 06:36:10.939332 7226 raft_consensus.cc:2781] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:10.939405 7226 raft_consensus.cc:482] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:10.939442 7226 raft_consensus.cc:3037] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.940186 7226 raft_consensus.cc:504] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.940342 7226 leader_election.cc:288] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 election: Requested vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591)
I20260810 06:36:10.940495 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b845ad7e04f401fa7f85751f8c71a04" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c36dc6d2e994e1ab6297854721540a0"
I20260810 06:36:10.940574 7086 raft_consensus.cc:3037] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.940526 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b845ad7e04f401fa7f85751f8c71a04" candidate_uuid: "68d1cadd43664f699fa014a56b708857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153"
I20260810 06:36:10.940610 6791 raft_consensus.cc:3037] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:10.941229 7086 raft_consensus.cc:2454] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 1.
I20260810 06:36:10.941422 6869 leader_election.cc:302] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68d1cadd43664f699fa014a56b708857, 8c36dc6d2e994e1ab6297854721540a0; no voters:
I20260810 06:36:10.941457 6791 raft_consensus.cc:2454] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d1cadd43664f699fa014a56b708857 in term 1.
I20260810 06:36:10.941531 7226 raft_consensus.cc:2781] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:10.941581 7226 raft_consensus.cc:686] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 1 LEADER]: Becoming Leader. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Running, Role: LEADER
I20260810 06:36:10.941638 7226 consensus_queue.cc:237] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.942345 6623 catalog_manager.cc:5495] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68d1cadd43664f699fa014a56b708857 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "68d1cadd43664f699fa014a56b708857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } health_report { overall_health: UNKNOWN } } }
I20260810 06:36:10.955631 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:10.953185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.30.71.2:41305)
0810 06:36:10.953236 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:10.953238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:10.953249 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:10.953309 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:10.953310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:10.953320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:10.953376 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:10.953545 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.954037 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.954812 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:10.955109 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:10.955197 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:10.955371 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:10.955406 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:10.955456 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:10.955523 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:36:10.955543 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:36:10.956596 7086 raft_consensus.cc:1261] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Refusing update from remote peer 68d1cadd43664f699fa014a56b708857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.956707 6791 raft_consensus.cc:1261] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Refusing update from remote peer 68d1cadd43664f699fa014a56b708857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:10.956844 7176 consensus_queue.cc:1035] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.957469 7176 consensus_queue.cc:1035] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:10.960162 6622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:10.960269 6622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:10.960917 6622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Sending DeleteTablet for 3 replicas of tablet 8b845ad7e04f401fa7f85751f8c71a04
I20260810 06:36:10.961213 6771 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b845ad7e04f401fa7f85751f8c71a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:34364
I20260810 06:36:10.961313 7198 tablet_replica.cc:330] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: stopping tablet replica
I20260810 06:36:10.961306 6918 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b845ad7e04f401fa7f85751f8c71a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:57970
I20260810 06:36:10.961366 7198 raft_consensus.cc:2229] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:10.961391 7197 tablet_replica.cc:330] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: stopping tablet replica
I20260810 06:36:10.961421 7198 raft_consensus.cc:2258] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.961355 6622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 06:36:10.961436 7197 raft_consensus.cc:2229] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:36:10.961488 7197 raft_consensus.cc:2258] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 06:36:10.961498 6622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:10.961817 7197 ts_tablet_manager.cc:1905] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.961999 7198 ts_tablet_manager.cc:1905] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.963573 7066 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b845ad7e04f401fa7f85751f8c71a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:10 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:43528
I20260810 06:36:10.963678 7196 tablet_replica.cc:330] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: stopping tablet replica
I20260810 06:36:10.963726 7196 raft_consensus.cc:2229] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:10.963768 7196 raft_consensus.cc:2258] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:10.964041 7198 ts_tablet_manager.cc:1918] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:10.964102 7198 log.cc:1196] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/8b845ad7e04f401fa7f85751f8c71a04
I20260810 06:36:10.964154 6918 tablet_service.cc:1467] Processing CreateTablet for tablet 29f1e53ccb0143a98a7d27fa22b60f8a (DEFAULT_TABLE table=test-table [id=a07aa18cada54eceab51ad54e34c041f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.964289 7066 tablet_service.cc:1467] Processing CreateTablet for tablet 29f1e53ccb0143a98a7d27fa22b60f8a (DEFAULT_TABLE table=test-table [id=a07aa18cada54eceab51ad54e34c041f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.964388 7198 ts_tablet_manager.cc:1939] T 8b845ad7e04f401fa7f85751f8c71a04 P b26bcda9d15344f9a6f472d1e0edb153: Deleting consensus metadata
I20260810 06:36:10.964402 7066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29f1e53ccb0143a98a7d27fa22b60f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.964102 6771 tablet_service.cc:1467] Processing CreateTablet for tablet 29f1e53ccb0143a98a7d27fa22b60f8a (DEFAULT_TABLE table=test-table [id=a07aa18cada54eceab51ad54e34c041f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:10.964707 6771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29f1e53ccb0143a98a7d27fa22b60f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.964720 6606 catalog_manager.cc:4841] TS b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533): tablet 8b845ad7e04f401fa7f85751f8c71a04 (table test-table [id=ba6423b294134baaa8eb341c710d05eb]) successfully deleted
I20260810 06:36:10.965554 7170 tablet_bootstrap.cc:492] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: Bootstrap starting.
I20260810 06:36:10.965620 7167 tablet_bootstrap.cc:492] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153: Bootstrap starting.
I20260810 06:36:10.965867 7170 tablet_bootstrap.cc:654] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.966146 7197 ts_tablet_manager.cc:1918] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:10.966202 7197 log.cc:1196] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/8b845ad7e04f401fa7f85751f8c71a04
I20260810 06:36:10.966274 7167 tablet_bootstrap.cc:654] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.966425 7197 ts_tablet_manager.cc:1939] T 8b845ad7e04f401fa7f85751f8c71a04 P 68d1cadd43664f699fa014a56b708857: Deleting consensus metadata
I20260810 06:36:10.966555 6918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29f1e53ccb0143a98a7d27fa22b60f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:10.967208 7167 tablet_bootstrap.cc:492] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153: No bootstrap required, opened a new log
I20260810 06:36:10.967278 7167 ts_tablet_manager.cc:1397] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:10.967447 7167 raft_consensus.cc:348] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.967545 7167 raft_consensus.cc:374] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.967574 7167 raft_consensus.cc:729] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26bcda9d15344f9a6f472d1e0edb153, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.967623 7167 consensus_queue.cc:260] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.967679 7168 tablet_bootstrap.cc:492] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: Bootstrap starting.
I20260810 06:36:10.967723 7167 ts_tablet_manager.cc:1428] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:10.967955 7170 tablet_bootstrap.cc:492] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: No bootstrap required, opened a new log
I20260810 06:36:10.967999 7170 ts_tablet_manager.cc:1397] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:10.968037 7168 tablet_bootstrap.cc:654] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:10.968096 7170 raft_consensus.cc:348] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.968154 7170 raft_consensus.cc:374] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.968173 7170 raft_consensus.cc:729] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c36dc6d2e994e1ab6297854721540a0, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.968214 7170 consensus_queue.cc:260] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.968302 7170 ts_tablet_manager.cc:1428] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:10.968698 6606 catalog_manager.cc:4841] TS 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305): tablet 8b845ad7e04f401fa7f85751f8c71a04 (table test-table [id=ba6423b294134baaa8eb341c710d05eb]) successfully deleted
I20260810 06:36:10.968832 7196 ts_tablet_manager.cc:1905] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:10.970429 7168 tablet_bootstrap.cc:492] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: No bootstrap required, opened a new log
I20260810 06:36:10.970489 7168 ts_tablet_manager.cc:1397] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:36:10.970559 7196 ts_tablet_manager.cc:1918] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:10.970619 7196 log.cc:1196] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/8b845ad7e04f401fa7f85751f8c71a04
I20260810 06:36:10.970652 7168 raft_consensus.cc:348] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.970727 7168 raft_consensus.cc:374] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:10.970757 7168 raft_consensus.cc:729] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d1cadd43664f699fa014a56b708857, State: Initialized, Role: FOLLOWER
I20260810 06:36:10.970799 7168 consensus_queue.cc:260] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:10.970835 7196 ts_tablet_manager.cc:1939] T 8b845ad7e04f401fa7f85751f8c71a04 P 8c36dc6d2e994e1ab6297854721540a0: Deleting consensus metadata
I20260810 06:36:10.970927 7168 ts_tablet_manager.cc:1428] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:10.971129 6607 catalog_manager.cc:4841] TS 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591): tablet 8b845ad7e04f401fa7f85751f8c71a04 (table test-table [id=ba6423b294134baaa8eb341c710d05eb]) successfully deleted
I20260810 06:36:11.048544 7192 raft_consensus.cc:482] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:11.048658 7192 raft_consensus.cc:504] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:11.048839 7192 leader_election.cc:288] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305)
I20260810 06:36:11.049139 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f1e53ccb0143a98a7d27fa22b60f8a" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153" is_pre_election: true
I20260810 06:36:11.049243 6791 raft_consensus.cc:2454] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 0.
I20260810 06:36:11.049193 6940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f1e53ccb0143a98a7d27fa22b60f8a" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68d1cadd43664f699fa014a56b708857" is_pre_election: true
I20260810 06:36:11.049340 6940 raft_consensus.cc:2454] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 0.
I20260810 06:36:11.049443 7017 leader_election.cc:302] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c36dc6d2e994e1ab6297854721540a0, b26bcda9d15344f9a6f472d1e0edb153; no voters:
I20260810 06:36:11.049547 7192 raft_consensus.cc:2781] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:11.049593 7192 raft_consensus.cc:482] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:11.049618 7192 raft_consensus.cc:3037] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:11.050345 7192 raft_consensus.cc:504] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:11.050480 7192 leader_election.cc:288] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 election: Requested vote from peers b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533), 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305)
I20260810 06:36:11.050638 6791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f1e53ccb0143a98a7d27fa22b60f8a" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26bcda9d15344f9a6f472d1e0edb153"
I20260810 06:36:11.050653 6940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f1e53ccb0143a98a7d27fa22b60f8a" candidate_uuid: "8c36dc6d2e994e1ab6297854721540a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68d1cadd43664f699fa014a56b708857"
I20260810 06:36:11.050720 6791 raft_consensus.cc:3037] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:11.050737 6940 raft_consensus.cc:3037] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:11.051445 6791 raft_consensus.cc:2454] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 1.
I20260810 06:36:11.051568 6940 raft_consensus.cc:2454] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c36dc6d2e994e1ab6297854721540a0 in term 1.
I20260810 06:36:11.051644 7017 leader_election.cc:302] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c36dc6d2e994e1ab6297854721540a0, b26bcda9d15344f9a6f472d1e0edb153; no voters:
I20260810 06:36:11.051744 7192 raft_consensus.cc:2781] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:11.051884 7192 raft_consensus.cc:686] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c36dc6d2e994e1ab6297854721540a0, State: Running, Role: LEADER
I20260810 06:36:11.051944 7192 consensus_queue.cc:237] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } }
I20260810 06:36:11.052578 6622 catalog_manager.cc:5495] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3). New cstate: current_term: 1 leader_uuid: "8c36dc6d2e994e1ab6297854721540a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c36dc6d2e994e1ab6297854721540a0" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46591 } health_report { overall_health: HEALTHY } } }
I20260810 06:36:11.066475 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:11.064470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43534 (local address 127.30.71.3:46591)
0810 06:36:11.064614 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:11.064616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:11.064624 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:11.064649 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:11.064650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:11.064665 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:11.064729 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:11.064814 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.065288 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.065971 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.066154 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.066203 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:11.066313 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:11.066338 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:11.066370 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:11.066404 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 06:36:11.066418 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":101,"threads_started":1}
I20260810 06:36:11.070926 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:11.068854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:46487 (local address 127.30.71.2:41305)
0810 06:36:11.068923 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:11.068925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:11.068992 (+ 67us) server_negotiation.cc:407] Connection header received
0810 06:36:11.069036 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:11.069037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:11.069045 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:11.069093 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:11.069195 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.069633 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.070331 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.070506 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.070554 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:11.070683 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:11.070706 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:11.070744 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:11.070827 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:36:11.070848 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:36:11.071013 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:11.068772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41305 (local address 127.30.71.3:46487)
0810 06:36:11.068923 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:11.068934 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:11.068996 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:11.069106 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:11.069107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:11.069108 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:36:11.069144 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:11.069146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.069693 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.069694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:11.070264 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:11.070266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.070569 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.070570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:11.070606 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:11.070619 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:36:11.070775 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:36:11.070777 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:36:11.070829 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:36:11.070894 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":95,"threads_started":1}
I20260810 06:36:11.073256 6940 raft_consensus.cc:1261] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Refusing update from remote peer 8c36dc6d2e994e1ab6297854721540a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:11.073292 6791 raft_consensus.cc:1261] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Refusing update from remote peer 8c36dc6d2e994e1ab6297854721540a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:11.073488 7192 consensus_queue.cc:1035] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26bcda9d15344f9a6f472d1e0edb153" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:11.073606 7254 consensus_queue.cc:1035] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68d1cadd43664f699fa014a56b708857" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:11.080461 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:11.078225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:52743 (local address 127.30.71.3:46591)
0810 06:36:11.078265 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:11.078267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:11.078278 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:11.078347 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:11.078348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:11.078360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:11.078416 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:11.078524 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.079066 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.079751 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.079939 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.080032 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:11.080203 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:11.080235 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:11.080290 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:11.080349 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 06:36:11.080375 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:36:11.080484 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:11.078139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46591 (local address 127.30.71.2:52743)
0810 06:36:11.078233 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:11.078245 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:11.078294 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:11.078428 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:11.078429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:11.078430 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:36:11.078472 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:11.078474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.079127 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.079128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:11.079681 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:11.079683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:11.080051 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:11.080052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:11.080113 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:11.080121 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:36:11.080297 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:36:11.080300 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:11.080348 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 06:36:11.080398 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:36:11.087239 6622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:11.087334 6622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:11.087983 6622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75ce6880f13c4dd1a38d9154c159ef60: Sending DeleteTablet for 3 replicas of tablet 29f1e53ccb0143a98a7d27fa22b60f8a
I20260810 06:36:11.088253 6771 tablet_service.cc:1514] Processing DeleteTablet for tablet 29f1e53ccb0143a98a7d27fa22b60f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:11 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:34364
I20260810 06:36:11.088291 6918 tablet_service.cc:1514] Processing DeleteTablet for tablet 29f1e53ccb0143a98a7d27fa22b60f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:11 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:57970
I20260810 06:36:11.088372 7198 tablet_replica.cc:330] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153: stopping tablet replica
I20260810 06:36:11.088395 7197 tablet_replica.cc:330] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: stopping tablet replica
I20260810 06:36:11.088454 7198 raft_consensus.cc:2229] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:11.088461 7197 raft_consensus.cc:2229] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:11.088495 7197 raft_consensus.cc:2258] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:11.088496 7198 raft_consensus.cc:2258] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:11.088728 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 6701
I20260810 06:36:11.088802 7197 ts_tablet_manager.cc:1905] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:11.088815 7198 ts_tablet_manager.cc:1905] T 29f1e53ccb0143a98a7d27fa22b60f8a P b26bcda9d15344f9a6f472d1e0edb153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:11.090590 7197 ts_tablet_manager.cc:1918] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:36:11.090673 7197 log.cc:1196] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/29f1e53ccb0143a98a7d27fa22b60f8a
I20260810 06:36:11.090889 7197 ts_tablet_manager.cc:1939] T 29f1e53ccb0143a98a7d27fa22b60f8a P 68d1cadd43664f699fa014a56b708857: Deleting consensus metadata
I20260810 06:36:11.091238 6606 catalog_manager.cc:4841] TS 68d1cadd43664f699fa014a56b708857 (127.30.71.2:41305): tablet 29f1e53ccb0143a98a7d27fa22b60f8a (table test-table [id=a07aa18cada54eceab51ad54e34c041f]) successfully deleted
I20260810 06:36:11.092846 7066 tablet_service.cc:1514] Processing DeleteTablet for tablet 29f1e53ccb0143a98a7d27fa22b60f8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:11 UTC) from {username='oryx', principal='oryx/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:43528
I20260810 06:36:11.092993 7196 tablet_replica.cc:330] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: stopping tablet replica
I20260810 06:36:11.093089 7196 raft_consensus.cc:2229] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:36:11.093183 7196 raft_consensus.cc:2258] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:11.093494 7196 ts_tablet_manager.cc:1905] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:11.095402 7196 ts_tablet_manager.cc:1918] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:36:11.095470 7196 log.cc:1196] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/29f1e53ccb0143a98a7d27fa22b60f8a
I20260810 06:36:11.095690 7196 ts_tablet_manager.cc:1939] T 29f1e53ccb0143a98a7d27fa22b60f8a P 8c36dc6d2e994e1ab6297854721540a0: Deleting consensus metadata
I20260810 06:36:11.096081 6607 catalog_manager.cc:4841] TS 8c36dc6d2e994e1ab6297854721540a0 (127.30.71.3:46591): tablet 29f1e53ccb0143a98a7d27fa22b60f8a (table test-table [id=a07aa18cada54eceab51ad54e34c041f]) successfully deleted
W20260810 06:36:11.097817 6606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:39533 (error 108)
I20260810 06:36:11.098531 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:11.098279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:39533 (local address 127.0.0.1:34380)
0810 06:36:11.098427 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:11.098480 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260810 06:36:11.098640 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 6851
W20260810 06:36:11.098663 6606 catalog_manager.cc:4584] TS b26bcda9d15344f9a6f472d1e0edb153 (127.30.71.1:39533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29f1e53ccb0143a98a7d27fa22b60f8a: Network error: Client connection negotiation failed: client connection to 127.30.71.1:39533: connect: Connection refused (error 111)
I20260810 06:36:11.109400 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 6999
I20260810 06:36:11.116484 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 6590
2026-08-10T06:36:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2963 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:36:11 dist-test-slave-w206 krb5kdc[7269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:36:11 dist-test-slave-w206 krb5kdc[7269](info): set up 2 sockets
Aug 10 06:36:11 dist-test-slave-w206 krb5kdc[7269](info): commencing operation
krb5kdc: starting...
W20260810 06:36:12.206749 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:36:12 dist-test-slave-w206 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:36:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:36:12.337939 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42193
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:42193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:12.422832 7283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:12.423187 7283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:12.423236 7283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:12.424626 7283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:12.424726 7283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:12.424753 7283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:12.424777 7283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:12.424810 7283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:12.426604 7283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:42193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:12.426954 7283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:12.427265 7283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:12.432845 7291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:12.432905 7289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:12.433141 7283 server_base.cc:1034] running on GCE node
W20260810 06:36:12.432848 7288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:12.433524 7283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:12.433845 7283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:12.435058 7283 hybrid_clock.cc:648] HybridClock initialized: now 1786343772434992 us; error 80 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T06:36:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1327 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:36:12 dist-test-slave-w206 krb5kdc[7297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:36:12 dist-test-slave-w206 krb5kdc[7297](info): set up 2 sockets
Aug 10 06:36:12 dist-test-slave-w206 krb5kdc[7297](info): commencing operation
krb5kdc: starting...
W20260810 06:36:13.542865 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:36:13 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:36:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:36:13.670657 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37605
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:13.753600 7311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:13.753948 7311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:13.753988 7311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:13.755194 7311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:13.755280 7311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:13.755304 7311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:13.755327 7311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:13.755373 7311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:13.756961 7311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:37605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:13.757225 7311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:13.757511 7311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:13.762795 7320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:13.762748 7317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:13.762748 7318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:13.762809 7311 server_base.cc:1034] running on GCE node
I20260810 06:36:13.763271 7311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:13.763535 7311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:13.764678 7311 hybrid_clock.cc:648] HybridClock initialized: now 1786343773764665 us; error 33 us; skew 500 ppm
Aug 10 06:36:13 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343773, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:13.772290 7311 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:36:13.772816 7311 webserver.cc:460] Webserver started at http://127.30.71.62:36995/ using document root <none> and password file <none>
I20260810 06:36:13.773092 7311 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:13.773142 7311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:13.773307 7311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:13.774436 7311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "05bc4aa447874c89b5d14f8d738e8e1b"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "c60bf5b13b7f06f023c0e4fbe118ef07"
server_key_iv: "ecda327866f10bbd4f6c5d4062822253"
server_key_version: "encryptionkey@0"
I20260810 06:36:13.774816 7311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "05bc4aa447874c89b5d14f8d738e8e1b"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "c60bf5b13b7f06f023c0e4fbe118ef07"
server_key_iv: "ecda327866f10bbd4f6c5d4062822253"
server_key_version: "encryptionkey@0"
I20260810 06:36:13.776269 7311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:13.777034 7327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:13.777267 7311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:36:13.777375 7311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "05bc4aa447874c89b5d14f8d738e8e1b"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "c60bf5b13b7f06f023c0e4fbe118ef07"
server_key_iv: "ecda327866f10bbd4f6c5d4062822253"
server_key_version: "encryptionkey@0"
I20260810 06:36:13.777467 7311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:13.788589 7311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:13.790586 7311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:13.790755 7311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:13.795047 7311 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:37605
I20260810 06:36:13.795068 7382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:37605 every 8 connection(s)
I20260810 06:36:13.795487 7311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:13.796041 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 7311
I20260810 06:36:13.796164 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:36:13.796365 31004 external_mini_cluster.cc:1428] Setting key ec21df9b11552cda09eaced1cb32c52d
I20260810 06:36:13.796512 7383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:13.799535 7383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Bootstrap starting.
I20260810 06:36:13.800189 7383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:13.800493 7383 log.cc:824] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:13.801287 7383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: No bootstrap required, opened a new log
Aug 10 06:36:13 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:13.803607 7383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:13.803779 7383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:13.803807 7383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05bc4aa447874c89b5d14f8d738e8e1b, State: Initialized, Role: FOLLOWER
I20260810 06:36:13.803965 7383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:13.804039 7383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:13.804073 7383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:13.804126 7383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:13.804865 7383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:13.804986 7383 leader_election.cc:302] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05bc4aa447874c89b5d14f8d738e8e1b; no voters:
I20260810 06:36:13.805176 7383 leader_election.cc:288] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:36:13.805228 7388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:13.805379 7388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 LEADER]: Becoming Leader. State: Replica: 05bc4aa447874c89b5d14f8d738e8e1b, State: Running, Role: LEADER
I20260810 06:36:13.805487 7383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:13.805473 7388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:13.805855 7388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05bc4aa447874c89b5d14f8d738e8e1b. Latest consensus state: current_term: 1 leader_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } } }
I20260810 06:36:13.805945 7388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:13.805863 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } } }
I20260810 06:36:13.806141 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:13.806159 7395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:13.806800 7395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:13.808275 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:13.796895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.30.71.62:37605)
0810 06:36:13.797113 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:13.797121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:13.797146 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:36:13.797613 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:13.797619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:13.797664 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:13.797813 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:13.798462 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:13.799427 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:13.800271 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:13.800763 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:13.800892 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:13.803657 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:13.803661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:13.803661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:13.803672 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:13.806117 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:13.806569 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:13.806572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:13.806572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:13.806672 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:13.807086 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:13.807088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:13.807088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:13.807367 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:13.807543 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:13.807914 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 06:36:13.808064 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260810 06:36:13.808868 7395 catalog_manager.cc:1347] Generated new cluster ID: 150fbfebf79d4d3cb9b2a96e8692363d
I20260810 06:36:13.808926 7395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:36:13.816999 7395 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:36:13.817713 7395 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:13.820930 7395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Generated new TSK 0
I20260810 06:36:13.821123 7395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:36:13.854092 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37605
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:13.960466 7409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:13.960668 7409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:13.960693 7409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:13.961887 7409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:13.961931 7409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:13.961958 7409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:13.963482 7409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:13.963724 7409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:13.963979 7409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:13.969944 7414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:13.969972 7415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:13.970014 7417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:13.970199 7409 server_base.cc:1034] running on GCE node
I20260810 06:36:13.970484 7409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:13.970762 7409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:13.971935 7409 hybrid_clock.cc:648] HybridClock initialized: now 1786343773971898 us; error 47 us; skew 500 ppm
Aug 10 06:36:13 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343773, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:13.974934 7409 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:36:13.975451 7409 webserver.cc:460] Webserver started at http://127.30.71.1:46437/ using document root <none> and password file <none>
I20260810 06:36:13.975713 7409 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:13.975782 7409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:13.975934 7409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:13.977005 7409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "f5ec0624c9f34b70b0257b7174f8ac76"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "8059ec3038cc68faf245fe22fc46fc3d"
server_key_iv: "8f4550630d2e51f2c74f75c7a21ea94b"
server_key_version: "encryptionkey@0"
I20260810 06:36:13.977377 7409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "f5ec0624c9f34b70b0257b7174f8ac76"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "8059ec3038cc68faf245fe22fc46fc3d"
server_key_iv: "8f4550630d2e51f2c74f75c7a21ea94b"
server_key_version: "encryptionkey@0"
I20260810 06:36:13.978724 7409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:13.979555 7425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:13.979735 7409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:13.979862 7409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "f5ec0624c9f34b70b0257b7174f8ac76"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "8059ec3038cc68faf245fe22fc46fc3d"
server_key_iv: "8f4550630d2e51f2c74f75c7a21ea94b"
server_key_version: "encryptionkey@0"
I20260810 06:36:13.979955 7409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:14.001497 7409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:14.003639 7409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:14.003821 7409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:14.004098 7409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:14.004433 7409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:14.004491 7409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.004534 7409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:14.004561 7409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.010754 7409 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:41759
I20260810 06:36:14.010802 7546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:41759 every 8 connection(s)
I20260810 06:36:14.011219 7409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343773, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:14.020246 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.012000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:39939 (local address 127.30.71.62:37605)
0810 06:36:14.012066 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.012072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.012269 (+ 197us) server_negotiation.cc:407] Connection header received
0810 06:36:14.012741 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.012741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.012751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.012813 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.013763 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.014259 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.015331 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.015536 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.015591 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.017698 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:14.017700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:14.017700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:14.017708 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:14.019052 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:14.019426 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:14.019426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:14.019427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:14.019479 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:14.019691 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:14.019691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:14.019691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:14.019794 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:14.019844 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:14.020095 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 06:36:14.020174 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:36:14.020413 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.011950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:39939)
0810 06:36:14.012131 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.012175 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.012658 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.012876 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.012879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.013478 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.013678 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.013683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.014351 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.014352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.015257 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.015259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.015615 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.015615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.015716 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.016199 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.016207 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.017599 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:14.019124 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:14.019130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:14.019132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:14.019359 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:14.019537 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:14.019537 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:14.019537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:14.019638 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:14.019852 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:14.019857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:14.019984 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 06:36:14.020066 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:36:14.020190 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260810 06:36:14.020696 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 7409
I20260810 06:36:14.020805 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:36:14.021056 31004 external_mini_cluster.cc:1428] Setting key aa73c61a12e642d0d86fd408d66cd617
I20260810 06:36:14.021046 7547 heartbeater.cc:347] Connected to a master server at 127.30.71.62:37605
I20260810 06:36:14.021143 7547 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:14.021373 7547 heartbeater.cc:511] Master 127.30.71.62:37605 requested a full tablet report, sending...
I20260810 06:36:14.021988 7344 ts_manager.cc:194] Registered new tserver with Master: f5ec0624c9f34b70b0257b7174f8ac76 (127.30.71.1:41759)
I20260810 06:36:14.022606 7344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:39939
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:36:14.047640 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37605
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:14.137533 7552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:14.137758 7552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:14.137786 7552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:14.139062 7552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:14.139114 7552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:14.139150 7552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:14.140691 7552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:14.140959 7552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:14.141223 7552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:14.146664 7557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.146665 7560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.146708 7552 server_base.cc:1034] running on GCE node
W20260810 06:36:14.146849 7558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.147181 7552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:14.147447 7552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:14.148634 7552 hybrid_clock.cc:648] HybridClock initialized: now 1786343774148614 us; error 36 us; skew 500 ppm
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:14.151268 7552 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:36:14.151762 7552 webserver.cc:460] Webserver started at http://127.30.71.2:44431/ using document root <none> and password file <none>
I20260810 06:36:14.151983 7552 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:14.152027 7552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:14.152119 7552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:14.153154 7552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "91a2245ac9934413954382ba9c540a5e"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "8d25a0b6779bde550515e5a54a8caf32"
server_key_iv: "17160219abde73e2cd9fd71bbe1924eb"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.153465 7552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "91a2245ac9934413954382ba9c540a5e"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "8d25a0b6779bde550515e5a54a8caf32"
server_key_iv: "17160219abde73e2cd9fd71bbe1924eb"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.154928 7552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:14.155750 7567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.155956 7552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:14.156055 7552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "91a2245ac9934413954382ba9c540a5e"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "8d25a0b6779bde550515e5a54a8caf32"
server_key_iv: "17160219abde73e2cd9fd71bbe1924eb"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.156143 7552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:14.169701 7552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:14.172369 7552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:14.172559 7552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:14.172823 7552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:14.173180 7552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:14.173239 7552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.173285 7552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:14.173309 7552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.179579 7552 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:43937
I20260810 06:36:14.179632 7690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:43937 every 8 connection(s)
I20260810 06:36:14.180027 7552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:36:14.183107 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 7552
I20260810 06:36:14.183300 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:36:14.183519 31004 external_mini_cluster.cc:1428] Setting key a70f8a9c5db1f47f2f3fcf8f60a68518
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:14.190958 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.180855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:35515 (local address 127.30.71.62:37605)
0810 06:36:14.180913 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.180915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.181163 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:36:14.181700 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.181702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.181715 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.181776 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.182763 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.183502 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.184869 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.185083 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.185164 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.187942 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:14.187945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:14.187945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:14.187953 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:14.189444 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:14.189895 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:14.189897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:14.189897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:14.189962 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:14.190229 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:14.190230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:14.190230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:14.190355 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:14.190467 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:14.190757 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 06:36:14.190846 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:36:14.191211 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.180857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:35515)
0810 06:36:14.181024 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.181063 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.181587 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.181848 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.181852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.182454 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.182653 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.182657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.183600 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.183601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.184767 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.184770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.185172 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.185173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.185308 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.185915 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.185928 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.187826 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:14.189556 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:14.189564 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:14.189566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:14.189804 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:14.190035 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:14.190036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:14.190036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:14.190158 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:14.190456 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:14.190464 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:14.190618 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 06:36:14.190734 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 06:36:14.190933 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260810 06:36:14.191664 7691 heartbeater.cc:347] Connected to a master server at 127.30.71.62:37605
I20260810 06:36:14.191762 7691 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:14.191967 7691 heartbeater.cc:511] Master 127.30.71.62:37605 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
I20260810 06:36:14.192416 7344 ts_manager.cc:194] Registered new tserver with Master: 91a2245ac9934413954382ba9c540a5e (127.30.71.2:43937)
I20260810 06:36:14.192796 7344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:35515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:36:14.210407 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37605
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:14.296530 7696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:14.296749 7696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:14.296782 7696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:14.297981 7696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:14.298026 7696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:14.298089 7696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:14.299613 7696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:14.299877 7696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:14.300132 7696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:14.305356 7704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.305384 7706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.305394 7696 server_base.cc:1034] running on GCE node
W20260810 06:36:14.305361 7701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.305838 7696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:14.306150 7696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:14.307312 7696 hybrid_clock.cc:648] HybridClock initialized: now 1786343774307322 us; error 62 us; skew 500 ppm
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:14.310060 7696 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:36:14.310523 7696 webserver.cc:460] Webserver started at http://127.30.71.3:39401/ using document root <none> and password file <none>
I20260810 06:36:14.310777 7696 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:14.310825 7696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:14.311020 7696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:14.312135 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "32ce11fbe99f41f4a8489b51757870d5"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "6a14106f494a6f4d25a87eedb9ba2185"
server_key_iv: "552d63816bae0473fe6b12c97263cc15"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.312512 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "32ce11fbe99f41f4a8489b51757870d5"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "6a14106f494a6f4d25a87eedb9ba2185"
server_key_iv: "552d63816bae0473fe6b12c97263cc15"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.313908 7696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:36:14.314685 7714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.314895 7696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:14.315022 7696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "32ce11fbe99f41f4a8489b51757870d5"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "6a14106f494a6f4d25a87eedb9ba2185"
server_key_iv: "552d63816bae0473fe6b12c97263cc15"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.315110 7696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:14.337993 7696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:14.340516 7696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:14.340693 7696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:14.340952 7696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:14.341282 7696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:14.341339 7696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.341391 7696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:14.341421 7696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.347396 7696 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:37175
I20260810 06:36:14.347492 7835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:37175 every 8 connection(s)
I20260810 06:36:14.347790 7696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:14.356336 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 7696
I20260810 06:36:14.356455 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:36:14.356652 31004 external_mini_cluster.cc:1428] Setting key 403e3a45636045670f8254c793900baf
I20260810 06:36:14.356978 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.348528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:43747 (local address 127.30.71.62:37605)
0810 06:36:14.348597 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.348600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.348813 (+ 213us) server_negotiation.cc:407] Connection header received
0810 06:36:14.349292 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.349293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.349303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.349357 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.350346 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.350845 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.351829 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.352029 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.352084 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.354127 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:14.354129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:14.354129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:14.354136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:14.355567 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:14.355930 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:14.355931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:14.355932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:14.355984 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:14.356250 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:14.356250 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:14.356251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:14.356353 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:14.356402 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:14.356761 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 06:36:14.356862 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:36:14.357174 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.348505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:43747)
0810 06:36:14.348703 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.348740 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.349207 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.349424 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.349428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.350017 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.350235 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.350240 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.350932 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.350933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.351755 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.351758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.352113 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.352114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.352214 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.352675 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.352683 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.354043 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:14.355636 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:14.355641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:14.355643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:14.355863 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:14.356042 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:14.356043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:14.356043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:14.356182 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:14.356420 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:14.356428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:14.356604 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 06:36:14.356726 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:36:14.356896 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":106,"threads_started":1}
I20260810 06:36:14.357707 7836 heartbeater.cc:347] Connected to a master server at 127.30.71.62:37605
I20260810 06:36:14.357825 7836 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:14.358003 7836 heartbeater.cc:511] Master 127.30.71.62:37605 requested a full tablet report, sending...
I20260810 06:36:14.358367 7344 ts_manager.cc:194] Registered new tserver with Master: 32ce11fbe99f41f4a8489b51757870d5 (127.30.71.3:37175)
I20260810 06:36:14.358747 7344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:43747
I20260810 06:36:14.368592 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab.
Entry for principal oryx/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab.
WARNING: no policy specified for oryx/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab.
Entry for principal oryx/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab.
WARNING: no policy specified for oryx/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab.
Entry for principal oryx/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab.
I20260810 06:36:14.448122 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 7409
I20260810 06:36:14.454375 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 7552
I20260810 06:36:14.461875 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 7696
I20260810 06:36:14.470222 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 7311
I20260810 06:36:14.476468 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37605
--webserver_interface=127.30.71.62
--webserver_port=36995
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:14.561761 7847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:14.562031 7847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:14.562071 7847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:14.563271 7847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:14.563354 7847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:14.563392 7847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:14.563426 7847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:14.563453 7847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:14.564973 7847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:37605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=36995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:14.565272 7847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:14.565506 7847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:14.570381 7853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.570389 7855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.570959 7852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.571013 7847 server_base.cc:1034] running on GCE node
I20260810 06:36:14.571370 7847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:14.571695 7847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:14.572898 7847 hybrid_clock.cc:648] HybridClock initialized: now 1786343774572887 us; error 49 us; skew 500 ppm
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:14.575556 7847 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:36:14.576035 7847 webserver.cc:460] Webserver started at http://127.30.71.62:36995/ using document root <none> and password file <none>
I20260810 06:36:14.576232 7847 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:14.576290 7847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:14.577625 7847 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:14.578282 7862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.578480 7847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:14.578574 7847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "05bc4aa447874c89b5d14f8d738e8e1b"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "c60bf5b13b7f06f023c0e4fbe118ef07"
server_key_iv: "ecda327866f10bbd4f6c5d4062822253"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.578926 7847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:14.609921 7847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:14.612576 7847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:14.612703 7847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:14.616732 7847 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:37605
I20260810 06:36:14.616890 7918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:37605 every 8 connection(s)
I20260810 06:36:14.617183 7847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:14.618753 7919 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:36:14.619521 7919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Bootstrap starting.
I20260810 06:36:14.622305 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 7847
I20260810 06:36:14.622641 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:41759
--local_ip_for_outbound_sockets=127.30.71.1
--tserver_master_addrs=127.30.71.62:37605
--webserver_port=46437
--webserver_interface=127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:36:14.628522 7919 log.cc:824] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:14.630280 7919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:36:14.630599 7919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Bootstrap complete.
I20260810 06:36:14.633080 7919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:14.633329 7919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05bc4aa447874c89b5d14f8d738e8e1b, State: Initialized, Role: FOLLOWER
I20260810 06:36:14.633497 7919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:14.633569 7919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:14.633635 7919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:14.633682 7919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:36:14.634734 7919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:14.634853 7919 leader_election.cc:302] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05bc4aa447874c89b5d14f8d738e8e1b; no voters:
I20260810 06:36:14.635059 7919 leader_election.cc:288] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:36:14.635113 7923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:36:14.635430 7919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:14.635434 7923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [term 2 LEADER]: Becoming Leader. State: Replica: 05bc4aa447874c89b5d14f8d738e8e1b, State: Running, Role: LEADER
I20260810 06:36:14.635619 7923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } }
I20260810 06:36:14.635938 7924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05bc4aa447874c89b5d14f8d738e8e1b. Latest consensus state: current_term: 2 leader_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } } }
I20260810 06:36:14.636109 7924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:14.635921 7925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bc4aa447874c89b5d14f8d738e8e1b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37605 } } }
I20260810 06:36:14.636406 7925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:14.636472 7933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:14.637107 7933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:14.637329 7933 catalog_manager.cc:1259] Loaded cluster ID: 150fbfebf79d4d3cb9b2a96e8692363d
I20260810 06:36:14.637537 7933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:36:14.638360 7933 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:14.638548 7933 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05bc4aa447874c89b5d14f8d738e8e1b: Loaded TSK: 0
I20260810 06:36:14.638775 7933 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:36:14.712165 7921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:14.712415 7921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:14.712458 7921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:14.713603 7921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:14.713670 7921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:14.713716 7921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:14.715232 7921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.1.keytab
--principal=oryx/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:41759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=46437
--tserver_master_addrs=127.30.71.62:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:14.715484 7921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:14.715718 7921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:14.720788 7945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.720783 7946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.721036 7921 server_base.cc:1034] running on GCE node
W20260810 06:36:14.720808 7949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.721345 7921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:14.721617 7921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:14.722779 7921 hybrid_clock.cc:648] HybridClock initialized: now 1786343774722763 us; error 34 us; skew 500 ppm
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:14.725396 7921 init.cc:377] Logged in from keytab as oryx/127.30.71.1@KRBTEST.COM (short username oryx)
I20260810 06:36:14.725906 7921 webserver.cc:460] Webserver started at http://127.30.71.1:46437/ using document root <none> and password file <none>
I20260810 06:36:14.726145 7921 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:14.726203 7921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:14.727649 7921 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:36:14.728294 7956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.728482 7921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:14.728574 7921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "f5ec0624c9f34b70b0257b7174f8ac76"
format_stamp: "Formatted at 2026-08-10 06:36:13 on dist-test-slave-w206"
server_key: "8059ec3038cc68faf245fe22fc46fc3d"
server_key_iv: "8f4550630d2e51f2c74f75c7a21ea94b"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.728940 7921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:14.738929 7921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:14.741294 7921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:14.741463 7921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:14.741688 7921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:14.742005 7921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:14.742062 7921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.742105 7921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:14.742130 7921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.747793 7921 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:41759
I20260810 06:36:14.747848 8080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:41759 every 8 connection(s)
I20260810 06:36:14.748173 7921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:14.748463 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 7921
I20260810 06:36:14.748813 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:43937
--local_ip_for_outbound_sockets=127.30.71.2
--tserver_master_addrs=127.30.71.62:37605
--webserver_port=44431
--webserver_interface=127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:14.759831 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.749022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:49295)
0810 06:36:14.749368 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.749410 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.750166 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.750800 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.750804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.751591 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.751871 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.751877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.753627 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.753628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.754876 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.754879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.755909 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.755910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.756041 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.756687 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.756697 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.759630 (+ 2933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":86,"threads_started":1}
W20260810 06:36:14.760179 8081 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:37605 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.71.62:37605: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
W20260810 06:36:14.760567 8085 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:14.749081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:49295 (local address 127.30.71.62:37605)
0810 06:36:14.749281 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.749290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.749514 (+ 224us) server_negotiation.cc:407] Connection header received
0810 06:36:14.750400 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.750409 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.750466 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.750577 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.752016 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.753457 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.755042 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.755688 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.755783 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.759780 (+ 3997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:49295: BlockingRecv error: recv got EOF from 127.30.71.1:49295 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:14.766274 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.760413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:43875)
0810 06:36:14.760828 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.760840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.760897 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.761038 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.761039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.761193 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.761243 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.761244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.762191 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.762193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.763078 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.763080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.763547 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.763548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.763612 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.764088 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.764093 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.766188 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396}
W20260810 06:36:14.766606 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:14.760490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:43875 (local address 127.30.71.62:37605)
0810 06:36:14.760631 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.760632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.760882 (+ 250us) server_negotiation.cc:407] Connection header received
0810 06:36:14.760947 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.760948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.760959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.761034 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.761327 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.762108 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.763160 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.763470 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.763539 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.766541 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:43875: BlockingRecv error: recv got EOF from 127.30.71.1:43875 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:14.772377 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.767122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:51537)
0810 06:36:14.767287 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.767299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.767347 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.767493 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.767494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.767657 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.767710 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.767712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.768649 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.768650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.769361 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.769364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.769829 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.769830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.769884 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.770392 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.770397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.772304 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
W20260810 06:36:14.772466 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:14.767182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:51537 (local address 127.30.71.62:37605)
0810 06:36:14.767218 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.767220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.767351 (+ 131us) server_negotiation.cc:407] Connection header received
0810 06:36:14.767396 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.767396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.767407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.767470 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.767788 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.768553 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.769450 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.769748 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.769819 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.772412 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:51537: BlockingRecv error: recv got EOF from 127.30.71.1:51537 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 06:36:14.772746 8081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:36:14.834890 8086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:14.835170 8086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:14.835197 8086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:14.836611 8086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:14.836653 8086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:14.836680 8086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:14.838160 8086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.2.keytab
--principal=oryx/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:43937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=44431
--tserver_master_addrs=127.30.71.62:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:14.838356 8086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:14.838526 8086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:14.843503 8093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.843475 8096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.843475 8094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.843727 8086 server_base.cc:1034] running on GCE node
I20260810 06:36:14.843971 8086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:14.844219 8086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:14.845391 8086 hybrid_clock.cc:648] HybridClock initialized: now 1786343774845376 us; error 27 us; skew 500 ppm
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:14.847956 8086 init.cc:377] Logged in from keytab as oryx/127.30.71.2@KRBTEST.COM (short username oryx)
I20260810 06:36:14.848439 8086 webserver.cc:460] Webserver started at http://127.30.71.2:44431/ using document root <none> and password file <none>
I20260810 06:36:14.848687 8086 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:14.848771 8086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:14.850100 8086 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:14.850701 8103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.850884 8086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:14.851009 8086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "91a2245ac9934413954382ba9c540a5e"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "8d25a0b6779bde550515e5a54a8caf32"
server_key_iv: "17160219abde73e2cd9fd71bbe1924eb"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.851392 8086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:14.863700 8086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:14.865669 8086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:14.865844 8086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:14.866079 8086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:14.866403 8086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:14.866460 8086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.866505 8086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:14.866532 8086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.872532 8086 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:43937
I20260810 06:36:14.872578 8222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:43937 every 8 connection(s)
I20260810 06:36:14.872931 8086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:36:14.874704 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 8086
I20260810 06:36:14.875108 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:37175
--local_ip_for_outbound_sockets=127.30.71.3
--tserver_master_addrs=127.30.71.62:37605
--webserver_port=39401
--webserver_interface=127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:36:14.881317 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:14.873644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:37379 (local address 127.30.71.62:37605)
0810 06:36:14.873688 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.873690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.873911 (+ 221us) server_negotiation.cc:407] Connection header received
0810 06:36:14.874364 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.874365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.874381 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.874436 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.875847 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.876341 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.877486 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.877705 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.877805 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.881223 (+ 3418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:37379: BlockingRecv error: recv got EOF from 127.30.71.2:37379 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:36:14.881706 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.873627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:37379)
0810 06:36:14.873765 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.873799 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.874275 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.874521 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.874525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.875444 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.875739 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.875746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.876446 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.876447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.877366 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.877369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.877817 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.877818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.877940 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.878512 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.878520 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.881458 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
W20260810 06:36:14.881995 8223 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:37605 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.71.62:37605: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:14.887111 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.882249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:40717)
0810 06:36:14.882299 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.882309 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.882372 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.882587 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.882588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.882704 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.882748 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.882749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.883717 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.883718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.884233 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.884234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.884764 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.884764 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.884815 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.885308 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.885313 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.887064 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:36:14.887229 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:14.882376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:40717 (local address 127.30.71.62:37605)
0810 06:36:14.882450 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.882453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.882462 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:14.882489 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.882491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.882505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.882567 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.882825 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.883637 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.884328 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.884676 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.884752 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.887167 (+ 2415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:40717: BlockingRecv error: recv got EOF from 127.30.71.2:40717 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:14.891851 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:14.887378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:43233)
0810 06:36:14.887524 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:14.887536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:14.887574 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:14.887718 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:14.887718 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:14.887825 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:14.887853 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.887854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.888745 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.888745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.889185 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:14.889186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.889638 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.889638 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:14.889676 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.890090 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:14.890094 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:14.891806 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
W20260810 06:36:14.891981 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:14.887484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:43233 (local address 127.30.71.62:37605)
0810 06:36:14.887551 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:14.887554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:14.887564 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:14.887643 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:14.887644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:14.887654 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:14.887705 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:14.887922 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.888674 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.889258 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:14.889571 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:14.889634 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:14.891910 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:43233: BlockingRecv error: recv got EOF from 127.30.71.2:43233 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 06:36:14.892004 8223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:36:14.961642 8226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:14.961854 8226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:14.961889 8226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:14.963020 8226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:14.963056 8226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:14.963083 8226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:14.964455 8226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/krb5kdc/oryx_127.30.71.3.keytab
--principal=oryx/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:37175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=39401
--tserver_master_addrs=127.30.71.62:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:14.964673 8226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:14.964845 8226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:14.969949 8232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:14.969985 8233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.970075 8226 server_base.cc:1034] running on GCE node
W20260810 06:36:14.970153 8235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:14.970427 8226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:14.970712 8226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:14.971865 8226 hybrid_clock.cc:648] HybridClock initialized: now 1786343774971854 us; error 28 us; skew 500 ppm
Aug 10 06:36:14 dist-test-slave-w206 krb5kdc[7297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343774, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:14.974478 8226 init.cc:377] Logged in from keytab as oryx/127.30.71.3@KRBTEST.COM (short username oryx)
I20260810 06:36:14.974941 8226 webserver.cc:460] Webserver started at http://127.30.71.3:39401/ using document root <none> and password file <none>
I20260810 06:36:14.975204 8226 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:14.975265 8226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:14.976567 8226 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:14.977106 8242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.977308 8226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:36:14.977380 8226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "32ce11fbe99f41f4a8489b51757870d5"
format_stamp: "Formatted at 2026-08-10 06:36:14 on dist-test-slave-w206"
server_key: "6a14106f494a6f4d25a87eedb9ba2185"
server_key_iv: "552d63816bae0473fe6b12c97263cc15"
server_key_version: "encryptionkey@0"
I20260810 06:36:14.977741 8226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:14.990757 8226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:14.992702 8226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:14.992866 8226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:14.993131 8226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:14.993489 8226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:14.993530 8226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.993554 8226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:14.993602 8226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:14.999815 8226 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:37175
I20260810 06:36:14.999862 8364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:37175 every 8 connection(s)
I20260810 06:36:15.000231 8226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:36:15.001293 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 8226
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:36:15.009267 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:15.001021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:54299 (local address 127.30.71.62:37605)
0810 06:36:15.001227 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:15.001229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:15.001370 (+ 141us) server_negotiation.cc:407] Connection header received
0810 06:36:15.002198 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:15.002199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:15.002210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:15.002281 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:15.003668 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.004408 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.005717 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.006010 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.006085 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.009194 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:54299: BlockingRecv error: recv got EOF from 127.30.71.3:54299 (error 108)
Metrics: {"server-negotiator.queue_time_us":174}
I20260810 06:36:15.009496 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:15.001006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:54299)
0810 06:36:15.001216 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:15.001281 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:15.002051 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:15.002380 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:15.002385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:15.003236 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:15.003552 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.003558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.004512 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.004514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.005634 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.005638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.006104 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.006105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.006215 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.006768 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:15.006776 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:15.009118 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
W20260810 06:36:15.009766 8365 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:37605 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.71.62:37605: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
I20260810 06:36:15.010375 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:15.002059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.30.71.62:37605)
0810 06:36:15.002105 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:15.002107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:15.002317 (+ 210us) server_negotiation.cc:407] Connection header received
0810 06:36:15.002332 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:15.002333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:15.002345 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:15.002415 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:15.003352 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.004125 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.004931 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.005237 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.005324 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.006700 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:15.006705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:15.006705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:15.006718 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:15.008482 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:15.008895 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:15.008899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:15.008899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:15.009008 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:15.009268 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:15.009270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:15.009270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:15.009658 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:15.009814 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:15.010138 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 06:36:15.010267 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:15.014413 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:15.010028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:57861)
0810 06:36:15.010078 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:15.010089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:15.010127 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:15.010261 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:15.010262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:15.010416 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:15.010678 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.010680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.011525 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.011526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.012016 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.012017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.012307 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.012308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.012350 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.012796 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:15.012806 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:15.014355 (+ 1549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 06:36:15.014478 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:15.010079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:57861 (local address 127.30.71.62:37605)
0810 06:36:15.010127 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:15.010129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:15.010138 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:15.010183 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:15.010183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:15.010195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:15.010255 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:15.010753 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.011459 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.012084 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.012253 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.012307 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.014437 (+ 2130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:57861: BlockingRecv error: recv got EOF from 127.30.71.3:57861 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:15.018414 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:15.014749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:35961)
0810 06:36:15.014803 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:15.014814 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:15.014859 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:15.014968 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:15.014969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:15.015126 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:15.015153 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.015154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.015699 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.015699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.016159 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.016160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.016456 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.016457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.016491 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.016896 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:15.016905 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:15.018364 (+ 1459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:36:15.018500 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:15.014821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:35961 (local address 127.30.71.62:37605)
0810 06:36:15.014874 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:15.014876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:15.014886 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:15.014905 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:15.014906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:15.014915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:15.014966 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:15.015213 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.015634 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.016225 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.016397 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.016444 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.018442 (+ 1998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:35961: BlockingRecv error: recv got EOF from 127.30.71.3:35961 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 06:36:15.018579 8365 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:15.778487 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:15.773132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:60791)
0810 06:36:15.773266 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:15.773278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:15.773343 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:15.773602 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:15.773603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:15.773780 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:15.773828 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.773830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.774641 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.774642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.775210 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.775212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.775582 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.775583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.775621 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.776268 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:15.776273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:15.778417 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
W20260810 06:36:15.778627 8370 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:15.773300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:60791 (local address 127.30.71.62:37605)
0810 06:36:15.773453 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:15.773455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:15.773463 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:15.773493 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:15.773493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:15.773504 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:15.773580 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:15.773949 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.774528 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.775299 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.775515 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.775576 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.778527 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:60791: BlockingRecv error: recv got EOF from 127.30.71.1:60791 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:15 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:15.897293 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:15.892481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:53115)
0810 06:36:15.892667 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:15.892679 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:15.892772 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:15.892926 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:15.892927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:15.893122 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:15.893171 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.893173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.893870 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.893871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.894435 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:15.894438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.894819 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.894820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:15.894879 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.895413 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:15.895419 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:15.897225 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
W20260810 06:36:15.897423 8370 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:15.892586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:53115 (local address 127.30.71.62:37605)
0810 06:36:15.892666 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:15.892668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:15.892752 (+ 84us) server_negotiation.cc:407] Connection header received
0810 06:36:15.892832 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:15.892833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:15.892847 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:15.892920 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:15.893259 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.893781 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.894533 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:15.894731 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:15.894820 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:15.897341 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:53115: BlockingRecv error: recv got EOF from 127.30.71.2:53115 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 06:36:16 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:16 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:16.023535 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:16.019035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:35745)
0810 06:36:16.019170 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:16.019181 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:16.019246 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:16.019372 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:16.019373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:16.019549 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:16.019598 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:16.019600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.020219 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.020219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:16.020756 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:16.020758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.021081 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.021082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:16.021120 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:16.021552 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:16.021557 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:16.023465 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
W20260810 06:36:16.023731 8370 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:16.019066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:35745 (local address 127.30.71.62:37605)
0810 06:36:16.019111 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:16.019113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:16.019242 (+ 129us) server_negotiation.cc:407] Connection header received
0810 06:36:16.019307 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:16.019308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:16.019320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:16.019370 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:16.019654 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.020150 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.020831 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.021014 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.021069 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:16.023639 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:35745: BlockingRecv error: recv got EOF from 127.30.71.3:35745 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 06:36:16 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:16 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:16.784299 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:16.779089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:37453)
0810 06:36:16.779259 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:16.779269 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:16.779322 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:16.779561 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:16.779562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:16.779730 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:16.779780 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:16.779782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.780627 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.780628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:16.781333 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:16.781336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.781844 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.781845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:16.781887 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:16.782394 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:16.782399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:16.784241 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
W20260810 06:36:16.784457 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:16.779270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:37453 (local address 127.30.71.62:37605)
0810 06:36:16.779421 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:16.779423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:16.779433 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:16.779458 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:16.779459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:16.779470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:16.779546 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:16.779856 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.780558 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.781553 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.781773 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.781833 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:16.784368 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:37453: BlockingRecv error: recv got EOF from 127.30.71.1:37453 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
Aug 10 06:36:16 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:16 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:16.902590 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:16.897772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:52923)
0810 06:36:16.897958 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:16.897969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:16.898060 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:16.898210 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:16.898211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:16.898401 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:16.898449 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:16.898451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.899356 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.899357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:16.899870 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:16.899872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.900201 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.900202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:16.900241 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:16.900766 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:16.900772 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:16.902534 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
W20260810 06:36:16.902676 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:16.897868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:52923 (local address 127.30.71.62:37605)
0810 06:36:16.897945 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:16.897948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:16.898043 (+ 95us) server_negotiation.cc:407] Connection header received
0810 06:36:16.898119 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:16.898120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:16.898133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:16.898205 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:16.898532 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.899280 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.899955 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:16.900135 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:16.900187 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:16.902627 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:52923: BlockingRecv error: recv got EOF from 127.30.71.2:52923 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 06:36:17 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:17 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:17.029124 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:17.023972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:56973)
0810 06:36:17.024165 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:17.024179 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:17.024255 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:17.024435 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:17.024436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:17.024622 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:17.024671 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:17.024673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.025347 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.025348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:17.025870 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:17.025872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.026263 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.026264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:17.026324 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:17.026787 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:17.026792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:17.029014 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
W20260810 06:36:17.029165 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:17.024109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:56973 (local address 127.30.71.62:37605)
0810 06:36:17.024179 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:17.024181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:17.024248 (+ 67us) server_negotiation.cc:407] Connection header received
0810 06:36:17.024319 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:17.024320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:17.024335 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:17.024382 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:17.024751 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.025256 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.025976 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.026175 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.026261 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:17.029097 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:56973: BlockingRecv error: recv got EOF from 127.30.71.3:56973 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 06:36:17 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:17 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:17.790033 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:17.784769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:58021)
0810 06:36:17.784915 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:17.784926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:17.784983 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:17.785176 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:17.785177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:17.785352 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:17.785403 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:17.785404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.786238 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.786239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:17.786761 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:17.786763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.787252 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.787253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:17.787301 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:17.787843 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:17.787848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:17.789962 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
W20260810 06:36:17.790237 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:17.784912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:58021 (local address 127.30.71.62:37605)
0810 06:36:17.785043 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:17.785045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:17.785059 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:36:17.785081 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:17.785082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:17.785092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:17.785155 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:17.785513 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.786108 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.786855 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.787126 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.787243 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:17.790141 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:58021: BlockingRecv error: recv got EOF from 127.30.71.1:58021 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
Aug 10 06:36:17 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:17 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:17.907917 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:17.903092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:50207)
0810 06:36:17.903317 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:17.903328 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:17.903420 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:17.903576 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:17.903577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:17.903772 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:17.903820 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:17.903822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.904492 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.904493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:17.905027 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:17.905029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.905410 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.905411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:17.905467 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:17.905923 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:17.905928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:17.907802 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":108,"threads_started":1}
W20260810 06:36:17.907946 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:17.903192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:50207 (local address 127.30.71.62:37605)
0810 06:36:17.903271 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:17.903273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:17.903402 (+ 129us) server_negotiation.cc:407] Connection header received
0810 06:36:17.903480 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:17.903481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:17.903495 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:17.903569 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:17.903912 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.904404 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.905122 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:17.905321 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:17.905410 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:17.907881 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:50207: BlockingRecv error: recv got EOF from 127.30.71.2:50207 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 06:36:18 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:18 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:18.035081 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:18.029712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:33589)
0810 06:36:18.030000 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:18.030013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:18.030114 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:18.030326 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:18.030327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:18.030516 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:18.030564 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:18.030566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.031375 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.031376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:18.032147 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:18.032150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.032578 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.032579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:18.032635 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:18.033094 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:18.033099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:18.034953 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":92,"threads_started":1}
W20260810 06:36:18.035123 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:18.029808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:33589 (local address 127.30.71.62:37605)
0810 06:36:18.029988 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:18.029990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:18.030093 (+ 103us) server_negotiation.cc:407] Connection header received
0810 06:36:18.030203 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:18.030204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:18.030225 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:18.030301 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:18.030648 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.031240 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.032219 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.032495 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.032567 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:18.035060 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:33589: BlockingRecv error: recv got EOF from 127.30.71.3:33589 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
Aug 10 06:36:18 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:18 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:18.798506 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:18.790569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:55007)
0810 06:36:18.790696 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:18.790706 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:18.790755 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:18.790875 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:18.790875 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:18.791098 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:18.791147 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:18.791149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.791790 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.791791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:18.792355 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:18.792356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.792711 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.792712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:18.792761 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:18.796431 (+ 3670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:18.796436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:18.798407 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
W20260810 06:36:18.798573 8383 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:18.790614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:55007 (local address 127.30.71.62:37605)
0810 06:36:18.790767 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:18.790769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:18.790777 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:18.790800 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:18.790801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:18.790811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:18.790872 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:18.791221 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.791718 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.792444 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.792634 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.792683 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:18.798498 (+ 5815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:55007: BlockingRecv error: recv got EOF from 127.30.71.1:55007 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
Aug 10 06:36:18 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:18 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:18.913487 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:18.908488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:50841)
0810 06:36:18.908683 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:18.908696 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:18.908767 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:18.908905 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:18.908906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:18.909084 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:18.909135 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:18.909136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.909940 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.909941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:18.910553 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:18.910556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.910920 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.910921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:18.910971 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:18.911553 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:18.911559 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:18.913417 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":118,"threads_started":1}
W20260810 06:36:18.913561 8383 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:18.908584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:50841 (local address 127.30.71.62:37605)
0810 06:36:18.908650 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:18.908652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:18.908769 (+ 117us) server_negotiation.cc:407] Connection header received
0810 06:36:18.908819 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:18.908820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:18.908832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:18.908899 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:18.909199 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.909700 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.910638 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:18.910821 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:18.910876 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:18.913512 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:50841: BlockingRecv error: recv got EOF from 127.30.71.2:50841 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 06:36:19 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:19 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:19.040686 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:19.035524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:41049)
0810 06:36:19.035666 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:19.035677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:19.035750 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:19.035909 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:19.035910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:19.036101 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:19.036152 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:19.036154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.036899 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.036900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:19.037532 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:19.037535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.038070 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.038071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:19.038133 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:19.038640 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:19.038645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:19.040625 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
W20260810 06:36:19.040817 8383 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:19.035590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:41049 (local address 127.30.71.62:37605)
0810 06:36:19.035669 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:19.035672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:19.035775 (+ 103us) server_negotiation.cc:407] Connection header received
0810 06:36:19.035803 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:19.035804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:19.035817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:19.035890 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:19.036286 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.036814 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.037657 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.037987 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.038057 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:19.040720 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:41049: BlockingRecv error: recv got EOF from 127.30.71.3:41049 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:36:19 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:19 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:19.803879 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:19.799001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:57755)
0810 06:36:19.799169 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:19.799180 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:19.799241 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:19.799379 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:19.799380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:19.799557 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:19.799606 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:19.799607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.800251 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.800251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:19.800803 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:19.800805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.801176 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.801177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:19.801234 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:19.801758 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:19.801763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:19.803793 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
W20260810 06:36:19.803951 8387 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:19.799126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:57755 (local address 127.30.71.62:37605)
0810 06:36:19.799256 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:19.799258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:19.799269 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:19.799293 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:19.799293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:19.799304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:19.799361 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:19.799674 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.800183 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.800882 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.801071 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.801125 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:19.803896 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:57755: BlockingRecv error: recv got EOF from 127.30.71.1:57755 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
Aug 10 06:36:19 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:19 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:19.919078 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:19.914157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:39363)
0810 06:36:19.914330 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:19.914344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:19.914414 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:19.914585 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:19.914586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:19.914775 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:19.914829 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:19.914831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.915500 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.915501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:19.916038 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:19.916039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.916363 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.916364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:19.916404 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:19.916863 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:19.916868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:19.919018 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":65,"threads_started":1}
W20260810 06:36:19.919190 8387 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:19.914376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:39363 (local address 127.30.71.62:37605)
0810 06:36:19.914450 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:19.914452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:19.914466 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:36:19.914494 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:19.914495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:19.914506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:19.914557 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:19.914901 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.915428 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.916119 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:19.916301 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:19.916349 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:19.919119 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:39363: BlockingRecv error: recv got EOF from 127.30.71.2:39363 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:36:20 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:20 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:20.046213 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:20.041227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:59883)
0810 06:36:20.041390 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:20.041411 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:20.041473 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:20.041602 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:20.041603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:20.041778 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:20.041829 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:20.041831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.042542 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.042544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:20.043136 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:20.043138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.043487 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.043488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:20.043530 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:20.044010 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:20.044015 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:20.046112 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
W20260810 06:36:20.046339 8387 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:20.041274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:59883 (local address 127.30.71.62:37605)
0810 06:36:20.041322 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:20.041324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:20.041485 (+ 161us) server_negotiation.cc:407] Connection header received
0810 06:36:20.041513 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:20.041514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:20.041527 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:20.041581 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:20.041897 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.042441 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.043219 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.043418 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.043471 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:20.046217 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:59883: BlockingRecv error: recv got EOF from 127.30.71.3:59883 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 10 06:36:20 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:20 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:20.809667 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:20.804499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:48029)
0810 06:36:20.804671 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:20.804682 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:20.804734 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:20.804983 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:20.804984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:20.805182 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:20.805261 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:20.805264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.806054 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.806056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:20.806608 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:20.806610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.806963 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.806964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:20.807064 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:20.807544 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:20.807549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:20.809601 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
W20260810 06:36:20.809826 8392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:20.804605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:48029 (local address 127.30.71.62:37605)
0810 06:36:20.804761 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:20.804763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:20.804774 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:20.804806 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:20.804807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:20.804818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:20.804950 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:20.805349 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.805978 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.806690 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.806897 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.806949 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:20.809726 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:48029: BlockingRecv error: recv got EOF from 127.30.71.1:48029 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
Aug 10 06:36:20 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:20 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:20.924314 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:20.919627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:47753)
0810 06:36:20.919798 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:20.919815 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:20.919876 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:20.920039 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:20.920039 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:20.920212 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:20.920261 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:20.920263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.920900 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.920901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:20.921429 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:20.921430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.921746 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.921746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:20.921783 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:20.922351 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:20.922355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:20.924225 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
W20260810 06:36:20.924468 8392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:20.919709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:47753 (local address 127.30.71.62:37605)
0810 06:36:20.919788 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:20.919790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:20.919881 (+ 91us) server_negotiation.cc:407] Connection header received
0810 06:36:20.919947 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:20.919948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:20.919959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:20.920013 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:20.920332 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.920830 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.921501 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:20.921680 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:20.921729 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:20.924398 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:47753: BlockingRecv error: recv got EOF from 127.30.71.2:47753 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 06:36:21 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:21 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:21.051889 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:21.046872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:52197)
0810 06:36:21.047013 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:21.047025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:21.047080 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:21.047250 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:21.047251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:21.047437 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:21.047485 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:21.047486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.048175 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.048176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:21.048885 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:21.048888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.049276 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.049277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:21.049336 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:21.049871 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:21.049877 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:21.051789 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
W20260810 06:36:21.051941 8392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:21.047060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:52197 (local address 127.30.71.62:37605)
0810 06:36:21.047135 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:21.047138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:21.047146 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:21.047171 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:21.047172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:21.047186 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:21.047233 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:21.047569 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.048077 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.048978 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.049193 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.049274 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:21.051873 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:52197: BlockingRecv error: recv got EOF from 127.30.71.3:52197 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:36:21 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:21 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:21.814796 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:21.810122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:33477)
0810 06:36:21.810282 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:21.810299 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:21.810348 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:21.810489 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:21.810490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:21.810658 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:21.810707 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:21.810708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.811414 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.811415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:21.811950 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:21.811952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.812283 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.812284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:21.812322 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:21.812811 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:21.812816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:21.814684 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
W20260810 06:36:21.814841 8396 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:21.810196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:33477 (local address 127.30.71.62:37605)
0810 06:36:21.810338 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:21.810340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:21.810348 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:21.810406 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:21.810407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:21.810417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:21.810466 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:21.810775 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.811336 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.812027 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.812216 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.812270 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:21.814770 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:33477: BlockingRecv error: recv got EOF from 127.30.71.1:33477 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
Aug 10 06:36:21 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:21 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:21.930213 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:21.925176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:59965)
0810 06:36:21.925374 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:21.925386 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:21.925493 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:21.925663 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:21.925664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:21.925851 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:21.925915 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:21.925917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.926606 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.926608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:21.927165 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:21.927167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.927575 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.927576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:21.927636 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:21.928099 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:21.928103 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:21.930116 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":128,"threads_started":1}
W20260810 06:36:21.930266 8396 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:21.925278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:59965 (local address 127.30.71.62:37605)
0810 06:36:21.925332 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:21.925334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:21.925473 (+ 139us) server_negotiation.cc:407] Connection header received
0810 06:36:21.925560 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:21.925561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:21.925574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:21.925643 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:21.926004 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.926515 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.927261 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:21.927481 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:21.927573 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:21.930202 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:59965: BlockingRecv error: recv got EOF from 127.30.71.2:59965 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 10 06:36:22 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:22 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:22.057265 8398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:22.052388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:58023)
0810 06:36:22.052564 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:22.052576 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:22.052640 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:22.052801 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:22.052802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:22.052979 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:22.053035 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:22.053037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.053693 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.053694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:22.054337 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:22.054339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.054666 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.054666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:22.054713 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:22.055254 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:22.055258 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:22.057165 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
W20260810 06:36:22.057312 8396 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:22.052444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:58023 (local address 127.30.71.62:37605)
0810 06:36:22.052514 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:22.052516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:22.052638 (+ 122us) server_negotiation.cc:407] Connection header received
0810 06:36:22.052693 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:22.052694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:22.052708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:22.052758 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:22.053100 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.053588 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.054413 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.054596 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.054648 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:22.057244 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:58023: BlockingRecv error: recv got EOF from 127.30.71.3:58023 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 06:36:22 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:22 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:22.820147 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:22.815382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:40661)
0810 06:36:22.815575 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:22.815594 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:22.815666 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:22.815805 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:22.815806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:22.815979 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:22.816028 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:22.816030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.816681 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.816682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:22.817238 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:22.817240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.817581 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.817582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:22.817620 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:22.818074 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:22.818078 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:22.820089 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
W20260810 06:36:22.820218 8401 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:22.815553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:40661 (local address 127.30.71.62:37605)
0810 06:36:22.815680 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:22.815682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:22.815690 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:36:22.815714 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:22.815715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:22.815725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:22.815783 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:22.816098 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.816610 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.817316 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.817518 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.817567 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:22.820165 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:40661: BlockingRecv error: recv got EOF from 127.30.71.1:40661 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
Aug 10 06:36:22 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:22 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:22.935667 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:22.930714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:39747)
0810 06:36:22.930867 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:22.930881 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:22.930956 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:22.931172 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:22.931173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:22.931385 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:22.931435 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:22.931436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.932324 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.932325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:22.932857 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:22.932858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.933179 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.933179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:22.933222 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:22.933683 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:22.933688 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:22.935577 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
W20260810 06:36:22.935798 8401 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:22.930815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:39747 (local address 127.30.71.62:37605)
0810 06:36:22.930870 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:22.930873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:22.930935 (+ 62us) server_negotiation.cc:407] Connection header received
0810 06:36:22.931068 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:22.931069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:22.931087 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:22.931154 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:22.931505 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.932251 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.932929 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:22.933110 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:22.933162 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:22.935736 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:39747: BlockingRecv error: recv got EOF from 127.30.71.2:39747 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 10 06:36:23 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:23 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:23.062568 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:23.057763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:48541)
0810 06:36:23.057934 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:23.057944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:23.058008 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:23.058150 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:23.058151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:23.058326 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:23.058377 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:23.058379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.059099 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.059100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:23.059691 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:23.059694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.060091 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.060092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:23.060151 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:23.060676 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:23.060682 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:23.062509 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
W20260810 06:36:23.062677 8401 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:23.057867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:48541 (local address 127.30.71.62:37605)
0810 06:36:23.057939 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:23.057941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:23.058003 (+ 62us) server_negotiation.cc:407] Connection header received
0810 06:36:23.058054 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:23.058055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:23.058070 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:23.058130 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:23.058446 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.059021 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.059784 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.059999 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.060089 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:23.062607 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:48541: BlockingRecv error: recv got EOF from 127.30.71.3:48541 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 06:36:23 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:23 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:23.831328 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:23.820622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:50403)
0810 06:36:23.820811 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:23.820823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:23.820874 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:23.821014 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:23.821015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:23.821186 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:23.821236 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:23.821237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.821905 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.821905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:23.822424 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:23.822425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.822814 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.822814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:23.822852 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:23.829104 (+ 6252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:23.829111 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:23.831230 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":99,"threads_started":1}
W20260810 06:36:23.831593 8405 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:23.820734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:50403 (local address 127.30.71.62:37605)
0810 06:36:23.820856 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:23.820857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:23.820866 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:23.820922 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:23.820923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:23.820933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:23.820989 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:23.821308 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.821827 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.822518 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.822737 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.822805 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:23.831395 (+ 8590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:50403: BlockingRecv error: recv got EOF from 127.30.71.1:50403 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
Aug 10 06:36:23 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:23 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:23.941457 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:23.936169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:52133)
0810 06:36:23.936319 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:23.936331 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:23.936413 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:23.936563 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:23.936564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:23.936740 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:23.936798 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:23.936800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.937591 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.937592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:23.938511 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:23.938515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.938856 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.938857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:23.938903 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:23.939438 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:23.939443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:23.941360 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
W20260810 06:36:23.941516 8405 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:23.936247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:52133 (local address 127.30.71.62:37605)
0810 06:36:23.936354 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:23.936356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:23.936407 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:36:23.936460 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:23.936461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:23.936473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:23.936521 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:23.936870 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.937508 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.938601 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:23.938783 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:23.938836 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:23.941454 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:52133: BlockingRecv error: recv got EOF from 127.30.71.2:52133 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 10 06:36:24 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:24 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:24.067670 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:24.063157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:33151)
0810 06:36:24.063315 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:24.063326 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:24.063396 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:24.063524 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:24.063525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:24.063696 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:24.063746 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:24.063747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.064357 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.064358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:24.064878 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:24.064879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.065238 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.065239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:24.065278 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:24.065800 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:24.065804 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:24.067611 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260810 06:36:24.067811 8405 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:24.063229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:33151 (local address 127.30.71.62:37605)
0810 06:36:24.063329 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:24.063331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:24.063373 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:36:24.063438 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:24.063439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:24.063451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:24.063503 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:24.063807 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.064293 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.064977 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.065162 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.065227 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:24.067724 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:33151: BlockingRecv error: recv got EOF from 127.30.71.3:33151 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 06:36:24 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:24 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:24.837044 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:24.831853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:46113)
0810 06:36:24.832015 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:24.832027 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:24.832081 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:24.832288 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:24.832289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:24.832521 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:24.832574 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:24.832576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.833230 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.833231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:24.833833 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:24.833835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.834180 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.834181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:24.834226 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:24.834736 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:24.834741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:24.836943 (+ 2202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260810 06:36:24.837102 8411 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:24.831938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:46113 (local address 127.30.71.62:37605)
0810 06:36:24.832103 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:24.832105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:24.832114 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:24.832137 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:24.832138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:24.832153 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:24.832248 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:24.832650 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.833156 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.833919 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.834107 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.834159 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:24.837029 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:46113: BlockingRecv error: recv got EOF from 127.30.71.1:46113 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
Aug 10 06:36:24 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:24 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:24.946486 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:24.941937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:57187)
0810 06:36:24.942117 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:24.942128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:24.942192 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:24.942331 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:24.942332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:24.942501 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:24.942553 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:24.942554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.943198 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.943199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:24.943706 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:24.943707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.944063 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.944064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:24.944118 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:24.944593 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:24.944598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:24.946428 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":110,"threads_started":1}
W20260810 06:36:24.946637 8411 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:24.942002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:57187 (local address 127.30.71.62:37605)
0810 06:36:24.942076 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:24.942079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:24.942195 (+ 116us) server_negotiation.cc:407] Connection header received
0810 06:36:24.942253 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:24.942253 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:24.942270 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:24.942312 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:24.942617 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.943135 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.943779 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:24.943968 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:24.944020 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:24.946552 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:57187: BlockingRecv error: recv got EOF from 127.30.71.2:57187 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 06:36:25 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:25 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:25.073120 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:25.068198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:57199)
0810 06:36:25.068381 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:25.068395 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:25.068486 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:25.068646 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:25.068647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:25.068873 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:25.068920 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:25.068923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.069599 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.069600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:25.070127 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:25.070130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.070526 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.070527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:25.070587 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:25.071273 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:25.071279 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:25.073020 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
W20260810 06:36:25.073179 8411 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:25.068321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:57199 (local address 127.30.71.62:37605)
0810 06:36:25.068384 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:25.068386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:25.068466 (+ 80us) server_negotiation.cc:407] Connection header received
0810 06:36:25.068552 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:25.068553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:25.068566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:25.068639 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:25.069017 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.069513 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.070228 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.070431 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.070524 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:25.073104 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:57199: BlockingRecv error: recv got EOF from 127.30.71.3:57199 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 06:36:25 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:25 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:25.842346 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:25.837568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:52947)
0810 06:36:25.837716 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:25.837733 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:25.837807 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:25.837984 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:25.837985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:25.838165 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:25.838212 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:25.838214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.838898 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.838899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:25.839486 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:25.839488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.839828 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.839829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:25.839872 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:25.840429 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:25.840434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:25.842283 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
W20260810 06:36:25.842461 8416 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:25.837654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:52947 (local address 127.30.71.62:37605)
0810 06:36:25.837813 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:25.837815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:25.837824 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:25.837886 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:25.837886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:25.837899 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:25.837963 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:25.838303 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.838823 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.839566 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.839759 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.839813 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:25.842384 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:52947: BlockingRecv error: recv got EOF from 127.30.71.1:52947 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
Aug 10 06:36:25 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:25 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:25.951929 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:25.947018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:42257)
0810 06:36:25.947161 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:25.947172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:25.947224 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:25.947408 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:25.947409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:25.947587 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:25.947634 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:25.947636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.948363 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.948363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:25.948889 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:25.948890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.949303 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.949304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:25.949348 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:25.949838 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:25.949842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:25.951819 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
W20260810 06:36:25.951987 8416 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:25.947127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:42257 (local address 127.30.71.62:37605)
0810 06:36:25.947256 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:25.947258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:25.947269 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:25.947308 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:25.947308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:25.947321 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:25.947393 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:25.947719 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.948276 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.948982 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:25.949193 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:25.949261 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:25.951923 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:42257: BlockingRecv error: recv got EOF from 127.30.71.2:42257 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 10 06:36:26 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:26 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:26.078424 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:26.073595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:36223)
0810 06:36:26.073779 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:26.073790 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:26.073859 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:26.074030 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:26.074031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:26.074250 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:26.074295 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:26.074297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.075011 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.075012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:26.075533 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:26.075534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.075902 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.075903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:26.075963 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:26.076480 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:26.076486 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:26.078318 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
W20260810 06:36:26.078490 8416 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:26.073692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:36223 (local address 127.30.71.62:37605)
0810 06:36:26.073772 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:26.073774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:26.073838 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:36:26.073939 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:26.073940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:26.073954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:26.074026 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:26.074396 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.074889 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.075623 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.075820 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.075902 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:26.078425 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:36223: BlockingRecv error: recv got EOF from 127.30.71.3:36223 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:36:26 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:26 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:26.847584 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:26.842969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:40249)
0810 06:36:26.843146 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:26.843163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:26.843227 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:26.843383 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:26.843384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:26.843562 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:26.843608 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:26.843609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.844253 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.844254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:26.844786 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:26.844787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.845130 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.845130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:26.845176 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:26.845730 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:26.845736 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:26.847515 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":121,"threads_started":1}
W20260810 06:36:26.847692 8420 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:26.843084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:40249 (local address 127.30.71.62:37605)
0810 06:36:26.843229 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:26.843231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:26.843267 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:36:26.843294 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:26.843294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:26.843305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:26.843358 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:26.843678 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.844171 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.844863 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.845054 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.845106 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:26.847611 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:40249: BlockingRecv error: recv got EOF from 127.30.71.1:40249 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
Aug 10 06:36:26 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:26 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:26.957779 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:26.952517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:40965)
0810 06:36:26.952753 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:26.952764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:26.952865 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:26.953038 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:26.953039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:26.953231 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:26.953277 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:26.953279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.954015 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.954016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:26.954540 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:26.954541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.954934 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.954935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:26.955019 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:26.955530 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:26.955541 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:26.957706 (+ 2165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":173,"threads_started":1}
W20260810 06:36:26.957906 8420 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:26.952652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:40965 (local address 127.30.71.62:37605)
0810 06:36:26.952744 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:26.952746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:26.952844 (+ 98us) server_negotiation.cc:407] Connection header received
0810 06:36:26.952946 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:26.952947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:26.952963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:26.953033 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:26.953413 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.953915 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.954634 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:26.954829 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:26.954934 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:26.957793 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:40965: BlockingRecv error: recv got EOF from 127.30.71.2:40965 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 06:36:27 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:27 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:27.084177 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:27.078920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:55421)
0810 06:36:27.079150 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:27.079163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:27.079262 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:27.079424 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:27.079426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:27.079620 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:27.079668 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:27.079670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.080411 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.080413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:27.080949 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:27.080952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.081399 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.081400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:27.081460 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:27.081974 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:27.081985 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:27.084104 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":127,"threads_started":1}
W20260810 06:36:27.084329 8420 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:27.079030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:55421 (local address 127.30.71.62:37605)
0810 06:36:27.079134 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:27.079137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:27.079239 (+ 102us) server_negotiation.cc:407] Connection header received
0810 06:36:27.079327 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:27.079328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:27.079342 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:27.079416 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:27.079763 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.080320 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.081047 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.081288 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.081357 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:27.084226 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:55421: BlockingRecv error: recv got EOF from 127.30.71.3:55421 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 06:36:27 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:27 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:27.853127 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:27.848215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:57533)
0810 06:36:27.848372 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:27.848385 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:27.848461 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:27.848654 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:27.848656 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:27.848910 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:27.848971 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:27.848973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.849662 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.849662 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:27.850185 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:27.850187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.850571 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.850572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:27.850611 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:27.851174 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:27.851179 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:27.853016 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
W20260810 06:36:27.853168 8424 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:27.848311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:57533 (local address 127.30.71.62:37605)
0810 06:36:27.848461 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:27.848464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:27.848474 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:27.848536 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:27.848537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:27.848551 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:27.848632 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:27.849053 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.849585 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.850294 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.850494 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.850558 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:27.853096 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:57533: BlockingRecv error: recv got EOF from 127.30.71.1:57533 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":57,"threads_started":1}
Aug 10 06:36:27 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:27 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:27.963063 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:27.958215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:36271)
0810 06:36:27.958357 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:27.958369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:27.958454 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:27.958613 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:27.958614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:27.958815 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:27.958860 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:27.958862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.959581 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.959582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:27.960126 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:27.960127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.960504 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.960505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:27.960562 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:27.961068 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:27.961074 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:27.962927 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
W20260810 06:36:27.963164 8424 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:27.958369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:36271 (local address 127.30.71.62:37605)
0810 06:36:27.958452 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:27.958455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:27.958465 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:27.958520 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:27.958521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:27.958534 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:27.958607 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:27.958963 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.959490 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.960219 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:27.960423 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:27.960505 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:27.963072 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:36271: BlockingRecv error: recv got EOF from 127.30.71.2:36271 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:36:28 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:28 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:28.089448 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:28.084704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:50269)
0810 06:36:28.084873 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:28.084883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:28.084943 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:28.085061 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:28.085062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:28.085239 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:28.085292 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:28.085295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.085904 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.085905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:28.086467 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:28.086470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.087046 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.087047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:28.087137 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:28.087596 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:28.087601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:28.089387 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
W20260810 06:36:28.089592 8424 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:28.084765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:50269 (local address 127.30.71.62:37605)
0810 06:36:28.084830 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:28.084832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:28.084946 (+ 114us) server_negotiation.cc:407] Connection header received
0810 06:36:28.084985 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:28.084985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:28.085000 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:28.085049 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:28.085354 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.085836 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.086559 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.086872 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.087018 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:28.089507 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:50269: BlockingRecv error: recv got EOF from 127.30.71.3:50269 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 06:36:28 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:28 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:28.861843 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:28.853596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:56259)
0810 06:36:28.853844 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:28.853859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:28.853918 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:28.854054 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:28.854055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:28.854263 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:28.854349 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:28.854352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.855115 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.855117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:28.855765 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:28.855768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.856304 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.856306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:28.856372 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:28.859914 (+ 3542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:28.859919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:28.861773 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":184,"threads_started":1}
W20260810 06:36:28.862049 8428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:28.853683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:56259 (local address 127.30.71.62:37605)
0810 06:36:28.853851 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:28.853853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:28.853903 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:36:28.853961 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:28.853962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:28.853972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:28.854034 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:28.854418 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.854942 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.855853 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.856191 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.856274 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:28.861860 (+ 5586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:56259: BlockingRecv error: recv got EOF from 127.30.71.1:56259 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
Aug 10 06:36:28 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:28 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:28.968758 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:28.963590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:57083)
0810 06:36:28.963795 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:28.963810 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:28.963906 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:28.964061 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:28.964062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:28.964254 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:28.964304 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:28.964306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.965207 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.965208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:28.965749 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:28.965750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.966223 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.966224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:28.966269 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:28.966790 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:28.966794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:28.968690 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
W20260810 06:36:28.968904 8428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:28.963778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:57083 (local address 127.30.71.62:37605)
0810 06:36:28.963850 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:28.963852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:28.963874 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:36:28.963972 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:28.963973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:28.963984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:28.964035 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:28.964391 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.965136 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.965845 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:28.966147 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:28.966209 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:28.968820 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:57083: BlockingRecv error: recv got EOF from 127.30.71.2:57083 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 06:36:29 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:29 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.3@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:29.094501 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:29.089891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.3:38349)
0810 06:36:29.090038 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:29.090051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:29.090115 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:29.090249 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:29.090250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:29.090429 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:29.090478 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:29.090479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.091131 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.091132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:29.091664 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:29.091665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.091998 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.091998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:29.092042 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:29.092577 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:29.092582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:29.094442 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
W20260810 06:36:29.094695 8428 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:29.089972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:38349 (local address 127.30.71.62:37605)
0810 06:36:29.090021 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:29.090027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:29.090116 (+ 89us) server_negotiation.cc:407] Connection header received
0810 06:36:29.090162 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:29.090163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:29.090175 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:29.090228 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:29.090544 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.091062 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.091733 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.091922 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.091971 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:29.094607 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.3:38349: BlockingRecv error: recv got EOF from 127.30.71.3:38349 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 06:36:29 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:29 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.1@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:29.867200 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:29.862448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.1:53077)
0810 06:36:29.862624 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:29.862635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:29.862693 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:29.862848 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:29.862849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:29.863039 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:29.863088 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:29.863090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.863757 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.863759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:29.864290 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:29.864292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.864620 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.864621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:29.864662 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:29.865136 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:29.865140 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:29.867118 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
W20260810 06:36:29.867321 8432 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:29.862537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:53077 (local address 127.30.71.62:37605)
0810 06:36:29.862674 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:29.862676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:29.862686 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:36:29.862739 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:29.862740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:29.862750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:29.862808 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:29.863161 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.863685 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.864363 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.864554 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.864604 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:29.867214 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.1:53077: BlockingRecv error: recv got EOF from 127.30.71.1:53077 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
Aug 10 06:36:29 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:36:29 dist-test-slave-w206 krb5kdc[7297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.71.2@KRBTEST.COM for oryx/127.30.71.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:36:29.974457 8433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:29.969360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37605 (local address 127.30.71.2:37375)
0810 06:36:29.969538 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:29.969552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:29.969630 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:29.969791 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:29.969792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:29.969987 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:29.970034 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:29.970036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.970763 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.970764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:29.971655 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:29.971659 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.972012 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.972013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:29.972052 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:29.972565 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:29.972570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:29.974399 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.71.62:37605: Server oryx/127.30.71.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":91,"threads_started":1}
W20260810 06:36:29.974654 8432 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:29.969468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:37375 (local address 127.30.71.62:37605)
0810 06:36:29.969542 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:29.969545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:29.969609 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:36:29.969692 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:29.969693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:29.969712 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:29.969773 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:29.970118 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.970631 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.971726 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:29.971935 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:29.972006 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:29.974562 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.71.2:37375: BlockingRecv error: recv got EOF from 127.30.71.2:37375 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:36:30.003598 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 7921
I20260810 06:36:30.010613 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 8086
I20260810 06:36:30.016944 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 8226
I20260810 06:36:30.022461 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 7847
2026-08-10T06:36:30Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17577 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T06:36:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:30Z Disabled control of system clock
I20260810 06:36:30.052625 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:45765
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:34849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:36:30.136216 8438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:30.136478 8438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:30.136518 8438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:30.137673 8438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:30.137712 8438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:30.137724 8438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:30.137737 8438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:30.137748 8438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:30.139060 8438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:45765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:30.139251 8438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:30.139498 8438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:30.144563 8443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.144613 8446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.144714 8444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.144745 8438 server_base.cc:1034] running on GCE node
I20260810 06:36:30.145116 8438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:30.145416 8438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:30.146587 8438 hybrid_clock.cc:648] HybridClock initialized: now 1786343790146565 us; error 53 us; skew 500 ppm
I20260810 06:36:30.147847 8438 webserver.cc:460] Webserver started at http://127.30.71.62:33401/ using document root <none> and password file <none>
I20260810 06:36:30.148089 8438 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:30.148159 8438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:30.148320 8438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:30.149385 8438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "b471b8cf829748f4b52b78c4a98f0a43"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "1b72954163b838e59d09adbaac2f3f4c"
server_key_iv: "ea7ed41442fc774136bea5e94ade0071"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.149748 8438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "b471b8cf829748f4b52b78c4a98f0a43"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "1b72954163b838e59d09adbaac2f3f4c"
server_key_iv: "ea7ed41442fc774136bea5e94ade0071"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.151198 8438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:30.151973 8453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.152176 8438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:30.152303 8438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "b471b8cf829748f4b52b78c4a98f0a43"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "1b72954163b838e59d09adbaac2f3f4c"
server_key_iv: "ea7ed41442fc774136bea5e94ade0071"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.152398 8438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:30.176230 8438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:30.176608 8438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:30.176782 8438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:30.181162 8438 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:45765
I20260810 06:36:30.181201 8507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:45765 every 8 connection(s)
I20260810 06:36:30.181560 8438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:30.182252 8508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:30.185060 8508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43: Bootstrap starting.
I20260810 06:36:30.185618 8508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:30.185859 8508 log.cc:824] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:30.186574 8508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43: No bootstrap required, opened a new log
I20260810 06:36:30.187397 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 8438
I20260810 06:36:30.187505 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:36:30.187709 31004 external_mini_cluster.cc:1428] Setting key 3158bf6b499212cfb723879086051566
I20260810 06:36:30.188820 8508 raft_consensus.cc:348] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471b8cf829748f4b52b78c4a98f0a43" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45765 } }
I20260810 06:36:30.188987 8508 raft_consensus.cc:374] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:30.189024 8508 raft_consensus.cc:729] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b471b8cf829748f4b52b78c4a98f0a43, State: Initialized, Role: FOLLOWER
I20260810 06:36:30.189198 8508 consensus_queue.cc:260] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471b8cf829748f4b52b78c4a98f0a43" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45765 } }
I20260810 06:36:30.189308 8508 raft_consensus.cc:388] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:30.189334 8508 raft_consensus.cc:482] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:30.189405 8508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:30.189949 8508 raft_consensus.cc:504] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471b8cf829748f4b52b78c4a98f0a43" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45765 } }
I20260810 06:36:30.190037 8508 leader_election.cc:302] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b471b8cf829748f4b52b78c4a98f0a43; no voters:
I20260810 06:36:30.190223 8508 leader_election.cc:288] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:36:30.190267 8513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:30.190466 8513 raft_consensus.cc:686] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [term 1 LEADER]: Becoming Leader. State: Replica: b471b8cf829748f4b52b78c4a98f0a43, State: Running, Role: LEADER
I20260810 06:36:30.190493 8508 sys_catalog.cc:564] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:30.190572 8513 consensus_queue.cc:237] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471b8cf829748f4b52b78c4a98f0a43" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45765 } }
I20260810 06:36:30.191005 8514 sys_catalog.cc:455] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b471b8cf829748f4b52b78c4a98f0a43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471b8cf829748f4b52b78c4a98f0a43" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45765 } } }
I20260810 06:36:30.191090 8514 sys_catalog.cc:458] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:30.191244 8516 sys_catalog.cc:455] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b471b8cf829748f4b52b78c4a98f0a43. Latest consensus state: current_term: 1 leader_uuid: "b471b8cf829748f4b52b78c4a98f0a43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471b8cf829748f4b52b78c4a98f0a43" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45765 } } }
I20260810 06:36:30.191293 8516 sys_catalog.cc:458] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:30.191954 8522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:30.192480 8522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:30.194176 8522 catalog_manager.cc:1347] Generated new cluster ID: 221501dbe790429dba4f970e634d6ea8
I20260810 06:36:30.194231 8522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:36:30.195294 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.188286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.30.71.62:45765)
0810 06:36:30.188525 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:30.188532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:30.188556 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:36:30.188801 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:30.188807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:30.188870 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:30.189006 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:30.189255 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.190878 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.192114 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.192754 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.192875 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:30.193679 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:30.193684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:30.193684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:30.193896 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:30.193938 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:30.193953 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:30.194026 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:30.195070 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0810 06:36:30.195141 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260810 06:36:30.216601 8522 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:36:30.217171 8522 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:30.225345 8522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b471b8cf829748f4b52b78c4a98f0a43: Generated new TSK 0
I20260810 06:36:30.225558 8522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:36:30.227154 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45765
--builtin_ntp_servers=127.30.71.20:34849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 06:36:30.312834 8536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:30.313092 8536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:30.313129 8536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:30.314307 8536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:30.314378 8536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:30.314420 8536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:30.315795 8536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:30.316068 8536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:30.316375 8536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:30.322599 8542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.322571 8543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.322770 8536 server_base.cc:1034] running on GCE node
W20260810 06:36:30.322577 8545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.327288 8536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:30.327587 8536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:30.328768 8536 hybrid_clock.cc:648] HybridClock initialized: now 1786343790328746 us; error 46 us; skew 500 ppm
I20260810 06:36:30.330214 8536 webserver.cc:460] Webserver started at http://127.30.71.1:36749/ using document root <none> and password file <none>
I20260810 06:36:30.330478 8536 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:30.330538 8536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:30.330668 8536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:30.332093 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "96f1baf203f243dca7ca52ff6781e3c8"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "8610c5fa5e8604f7de7380c43a3517ae"
server_key_iv: "8be4e592e14d2c027237b6ac77d33a6d"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.332520 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "96f1baf203f243dca7ca52ff6781e3c8"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "8610c5fa5e8604f7de7380c43a3517ae"
server_key_iv: "8be4e592e14d2c027237b6ac77d33a6d"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.333803 8536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 06:36:30.334519 8551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.334705 8536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.334810 8536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "96f1baf203f243dca7ca52ff6781e3c8"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "8610c5fa5e8604f7de7380c43a3517ae"
server_key_iv: "8be4e592e14d2c027237b6ac77d33a6d"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.334904 8536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:30.348258 8536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:30.348585 8536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:30.348742 8536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:30.348978 8536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:30.349313 8536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:30.349368 8536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.349438 8536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:30.349470 8536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.355523 8536 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:37629
I20260810 06:36:30.355595 8672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:37629 every 8 connection(s)
I20260810 06:36:30.355928 8536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:30.361011 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.356722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:38339 (local address 127.30.71.62:45765)
0810 06:36:30.356786 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:30.356787 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:30.357011 (+ 224us) server_negotiation.cc:407] Connection header received
0810 06:36:30.357482 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:30.357483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:30.357496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:30.357548 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:30.358162 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.358649 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.359695 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.359910 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.359967 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:30.360764 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:30.360765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:30.360765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:30.360770 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:30.360783 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:30.360787 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:30.360830 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:30.360922 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 06:36:30.360949 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:36:30.361179 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.356717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45765 (local address 127.30.71.1:38339)
0810 06:36:30.356893 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:30.356926 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:30.357402 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:30.357602 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:30.357606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:30.357613 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:30.358093 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:30.358097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.358728 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.358729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:30.359617 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:30.359620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.359988 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.359989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:30.360098 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:30.360638 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:36:30.360638 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:30.360665 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:36:30.360686 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:30.360842 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:30.360846 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:30.360849 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:30.360907 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:36:30.360985 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":94,"threads_started":1}
I20260810 06:36:30.361720 8673 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45765
I20260810 06:36:30.361841 8673 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:30.362018 8673 heartbeater.cc:511] Master 127.30.71.62:45765 requested a full tablet report, sending...
I20260810 06:36:30.362550 8471 ts_manager.cc:194] Registered new tserver with Master: 96f1baf203f243dca7ca52ff6781e3c8 (127.30.71.1:37629)
I20260810 06:36:30.362797 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 8536
I20260810 06:36:30.362888 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:36:30.363113 31004 external_mini_cluster.cc:1428] Setting key ac3aefd074ac2eddf459aaee101f3d84
I20260810 06:36:30.363325 8471 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.1:38339
I20260810 06:36:30.364194 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45765
--builtin_ntp_servers=127.30.71.20:34849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 06:36:30.446539 8676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:30.446744 8676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:30.446765 8676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:30.447901 8676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:30.447939 8676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:30.447963 8676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:30.449221 8676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:30.449414 8676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:30.449657 8676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:30.454751 8687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.454752 8682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.454800 8684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.455127 8676 server_base.cc:1034] running on GCE node
I20260810 06:36:30.455363 8676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:30.455632 8676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:30.456810 8676 hybrid_clock.cc:648] HybridClock initialized: now 1786343790456825 us; error 56 us; skew 500 ppm
I20260810 06:36:30.458076 8676 webserver.cc:460] Webserver started at http://127.30.71.2:40371/ using document root <none> and password file <none>
I20260810 06:36:30.458348 8676 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:30.458424 8676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:30.458581 8676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:30.459785 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "b7c784a63fa54d2fb3f41372ee2b70dc"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "4421d7849951a4b2fd1c5c95d073ea24"
server_key_iv: "359017106aa99d41ae45dc480918b769"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.460168 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "b7c784a63fa54d2fb3f41372ee2b70dc"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "4421d7849951a4b2fd1c5c95d073ea24"
server_key_iv: "359017106aa99d41ae45dc480918b769"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.461508 8676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:36:30.462198 8693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.462371 8676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:30.462477 8676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "b7c784a63fa54d2fb3f41372ee2b70dc"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "4421d7849951a4b2fd1c5c95d073ea24"
server_key_iv: "359017106aa99d41ae45dc480918b769"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.462553 8676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:30.483956 8676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:30.484320 8676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:30.484485 8676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:30.484741 8676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:30.485121 8676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:30.485179 8676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.485216 8676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:30.485270 8676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.491294 8676 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:40441
I20260810 06:36:30.491338 8817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:40441 every 8 connection(s)
I20260810 06:36:30.491739 8676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:36:30.496838 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.492562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:57397 (local address 127.30.71.62:45765)
0810 06:36:30.492632 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:30.492634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:30.492862 (+ 228us) server_negotiation.cc:407] Connection header received
0810 06:36:30.493349 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:30.493350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:30.493360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:30.493425 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:30.494082 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.494613 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.495615 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.495801 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.495851 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:30.496604 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:30.496605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:30.496605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:30.496610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:30.496624 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:30.496628 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:30.496672 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:30.496751 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:36:30.496777 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:36:30.497056 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.492545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45765 (local address 127.30.71.2:57397)
0810 06:36:30.492726 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:30.492767 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:30.493261 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:30.493478 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:30.493481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:30.493489 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:30.494002 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:30.494006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.494706 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.494707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:30.495531 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:30.495533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.495873 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.495874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:30.495982 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:30.496475 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:36:30.496475 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:30.496504 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:36:30.496525 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:30.496685 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:30.496689 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:30.496691 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:36:30.496742 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:36:30.496818 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260810 06:36:30.497583 8818 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45765
I20260810 06:36:30.497663 8818 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:30.497865 8818 heartbeater.cc:511] Master 127.30.71.62:45765 requested a full tablet report, sending...
I20260810 06:36:30.498250 8470 ts_manager.cc:194] Registered new tserver with Master: b7c784a63fa54d2fb3f41372ee2b70dc (127.30.71.2:40441)
I20260810 06:36:30.498592 8470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.2:57397
I20260810 06:36:30.499459 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 8676
I20260810 06:36:30.499538 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:36:30.499759 31004 external_mini_cluster.cc:1428] Setting key 6e0bfdaeb37b8e98d73676bffa59c00e
I20260810 06:36:30.500854 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45765
--builtin_ntp_servers=127.30.71.20:34849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 06:36:30.589378 8821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:30.589576 8821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:30.589597 8821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:30.590950 8821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:30.591001 8821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:30.591048 8821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:30.592516 8821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:34849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:30.592773 8821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:30.593093 8821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:30.598270 8829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.598248 8827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.598546 8821 server_base.cc:1034] running on GCE node
W20260810 06:36:30.598682 8826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.598966 8821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:30.599282 8821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:30.600430 8821 hybrid_clock.cc:648] HybridClock initialized: now 1786343790600414 us; error 28 us; skew 500 ppm
I20260810 06:36:30.601768 8821 webserver.cc:460] Webserver started at http://127.30.71.3:35235/ using document root <none> and password file <none>
I20260810 06:36:30.602036 8821 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:30.602084 8821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:30.602260 8821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:30.603428 8821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "d640aec6d43346b6ae1d1e007a5baaf6"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "d7edb7bff1fdb8dbebaffda9db8a7e3d"
server_key_iv: "f8561d098b486b1f07cb54790192e219"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.603788 8821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "d640aec6d43346b6ae1d1e007a5baaf6"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "d7edb7bff1fdb8dbebaffda9db8a7e3d"
server_key_iv: "f8561d098b486b1f07cb54790192e219"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.605134 8821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:36:30.605916 8835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.606132 8821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:36:30.606236 8821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "d640aec6d43346b6ae1d1e007a5baaf6"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "d7edb7bff1fdb8dbebaffda9db8a7e3d"
server_key_iv: "f8561d098b486b1f07cb54790192e219"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.606315 8821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:30.619819 8821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:30.620118 8821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:30.620227 8821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:30.620415 8821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:30.620754 8821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:30.620788 8821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.620844 8821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:30.620877 8821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.626941 8821 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:35045
I20260810 06:36:30.627039 8959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:35045 every 8 connection(s)
I20260810 06:36:30.627379 8821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:36:30.632452 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.628165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:33035 (local address 127.30.71.62:45765)
0810 06:36:30.628237 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:30.628239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:30.628475 (+ 236us) server_negotiation.cc:407] Connection header received
0810 06:36:30.628958 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:30.628959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:30.628969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:30.629023 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:30.629663 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.630153 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.631180 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.631385 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.631440 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:30.632209 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:30.632211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:30.632211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:30.632215 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:30.632229 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:30.632232 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:30.632275 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:30.632366 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 06:36:30.632391 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:36:30.632649 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.628143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45765 (local address 127.30.71.3:33035)
0810 06:36:30.628350 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:30.628386 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:30.628861 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:30.629078 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:30.629081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:30.629090 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:30.629594 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:30.629598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.630236 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.630236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:30.631107 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:30.631110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:30.631461 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:30.631462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:30.631589 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:30.632082 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:36:30.632083 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:30.632111 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:36:30.632133 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:30.632287 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:30.632290 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:30.632293 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:30.632344 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 06:36:30.632424 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260810 06:36:30.633204 8960 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45765
I20260810 06:36:30.633285 8960 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:30.633494 8960 heartbeater.cc:511] Master 127.30.71.62:45765 requested a full tablet report, sending...
I20260810 06:36:30.633891 8470 ts_manager.cc:194] Registered new tserver with Master: d640aec6d43346b6ae1d1e007a5baaf6 (127.30.71.3:35045)
I20260810 06:36:30.634227 8470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.3:33035
I20260810 06:36:30.636699 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 8821
I20260810 06:36:30.636775 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:36:30.636992 31004 external_mini_cluster.cc:1428] Setting key fdc79d95dbd792f1c185d783f1a05417
I20260810 06:36:30.637908 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:36:30.639784 8512 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:30.639276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53184 (local address 127.30.71.62:45765)
0810 06:36:30.639318 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:30.639322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:30.639476 (+ 154us) server_negotiation.cc:407] Connection header received
0810 06:36:30.639500 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:30.639500 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:30.639514 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:30.639560 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:30.639710 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53184: BlockingRecv error: recv got EOF from 127.0.0.1:53184 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:36:30.639823 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 8536
I20260810 06:36:30.647531 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 8676
I20260810 06:36:30.653510 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 8821
I20260810 06:36:30.659574 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 8438
2026-08-10T06:36:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (637 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T06:36:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:30Z Disabled control of system clock
I20260810 06:36:30.690187 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42089
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:42089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:36:30.771514 8974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:30.771739 8974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:30.771770 8974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:30.772881 8974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:30.772917 8974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:30.772930 8974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:30.772943 8974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:30.772954 8974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:30.774323 8974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:42089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:30.774535 8974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:30.774799 8974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:30.779997 8980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.779991 8982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.780081 8979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.780282 8974 server_base.cc:1034] running on GCE node
I20260810 06:36:30.780634 8974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:30.780895 8974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:30.782069 8974 hybrid_clock.cc:648] HybridClock initialized: now 1786343790782056 us; error 27 us; skew 500 ppm
I20260810 06:36:30.783384 8974 webserver.cc:460] Webserver started at http://127.30.71.62:38759/ using document root <none> and password file <none>
I20260810 06:36:30.783641 8974 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:30.783713 8974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:30.783865 8974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:30.784997 8974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "a1c3a912d02e4119ab5da728c0114352"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "c88b8bd9a9bff167ea4db83798deaf58"
server_key_iv: "10ebf01959973d846338bb12f6dda4eb"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.785382 8974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "a1c3a912d02e4119ab5da728c0114352"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "c88b8bd9a9bff167ea4db83798deaf58"
server_key_iv: "10ebf01959973d846338bb12f6dda4eb"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.786741 8974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 06:36:30.787655 8988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.787853 8974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:36:30.787966 8974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "a1c3a912d02e4119ab5da728c0114352"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "c88b8bd9a9bff167ea4db83798deaf58"
server_key_iv: "10ebf01959973d846338bb12f6dda4eb"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.788053 8974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:30.791471 8974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:30.791745 8974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:30.791898 8974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:30.795737 8974 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:42089
I20260810 06:36:30.795778 9044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:42089 every 8 connection(s)
I20260810 06:36:30.796144 8974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:30.796866 9045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:30.799618 9045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352: Bootstrap starting.
I20260810 06:36:30.800150 9045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:30.800400 9045 log.cc:824] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:30.801084 9045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352: No bootstrap required, opened a new log
I20260810 06:36:30.802620 9045 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c3a912d02e4119ab5da728c0114352" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42089 } }
I20260810 06:36:30.802757 9045 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:30.802779 9045 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c3a912d02e4119ab5da728c0114352, State: Initialized, Role: FOLLOWER
I20260810 06:36:30.802917 9045 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c3a912d02e4119ab5da728c0114352" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42089 } }
I20260810 06:36:30.803045 9045 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:30.803089 9045 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:30.803144 9045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:30.803696 9045 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c3a912d02e4119ab5da728c0114352" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42089 } }
I20260810 06:36:30.803825 9045 leader_election.cc:302] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1c3a912d02e4119ab5da728c0114352; no voters:
I20260810 06:36:30.804008 9045 leader_election.cc:288] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:36:30.804087 9049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:30.804246 9049 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [term 1 LEADER]: Becoming Leader. State: Replica: a1c3a912d02e4119ab5da728c0114352, State: Running, Role: LEADER
I20260810 06:36:30.804279 9045 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:30.804338 9049 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c3a912d02e4119ab5da728c0114352" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42089 } }
I20260810 06:36:30.804697 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 8974
I20260810 06:36:30.804826 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:36:30.805076 31004 external_mini_cluster.cc:1428] Setting key e2a1a1f38395db4dc067921db2f48572
I20260810 06:36:30.805044 9051 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1c3a912d02e4119ab5da728c0114352. Latest consensus state: current_term: 1 leader_uuid: "a1c3a912d02e4119ab5da728c0114352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c3a912d02e4119ab5da728c0114352" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42089 } } }
I20260810 06:36:30.805115 9051 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:30.805454 9050 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1c3a912d02e4119ab5da728c0114352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c3a912d02e4119ab5da728c0114352" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42089 } } }
I20260810 06:36:30.805502 9058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:30.805563 9050 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352 [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:30.806114 9058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:30.807790 9058 catalog_manager.cc:1347] Generated new cluster ID: d9c699025f1f4d1681953dd1cb143f45
I20260810 06:36:30.807843 9058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:36:30.808085 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.805879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.30.71.62:42089)
0810 06:36:30.806095 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:30.806105 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:30.806137 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:36:30.806265 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:30.806272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:30.806318 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:30.806461 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:30.807545 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:30.807549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:30.807549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:30.807554 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:30.807600 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:30.807616 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:30.807688 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:30.807879 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 06:36:30.807939 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":64,"threads_started":1}
I20260810 06:36:30.817013 9058 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:36:30.817206 9058 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:30.826320 9058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1c3a912d02e4119ab5da728c0114352: Generated new TSK 0
I20260810 06:36:30.826568 9058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:36:30.828399 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42089
--builtin_ntp_servers=127.30.71.20:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:36:30.911695 9071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:30.911942 9071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:30.911984 9071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:30.913153 9071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:30.913221 9071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:30.913268 9071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:30.914635 9071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:30.914865 9071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:30.915210 9071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:30.921056 9076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.921067 9079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:30.921062 9077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:30.921351 9071 server_base.cc:1034] running on GCE node
I20260810 06:36:30.921613 9071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:30.921869 9071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:30.923039 9071 hybrid_clock.cc:648] HybridClock initialized: now 1786343790923020 us; error 40 us; skew 500 ppm
I20260810 06:36:30.924326 9071 webserver.cc:460] Webserver started at http://127.30.71.1:45243/ using document root <none> and password file <none>
I20260810 06:36:30.924606 9071 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:30.924676 9071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:30.924822 9071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:30.925889 9071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "cc6e86a7e732400a8e5f8c1fb0ae08f1"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "50b192c101ff80be9f9b8c61d4695ce9"
server_key_iv: "9de826fdd777ae8cc73f8a3be4d40def"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.926258 9071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "cc6e86a7e732400a8e5f8c1fb0ae08f1"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "50b192c101ff80be9f9b8c61d4695ce9"
server_key_iv: "9de826fdd777ae8cc73f8a3be4d40def"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.927735 9071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:36:30.928577 9085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.928771 9071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:36:30.928875 9071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "cc6e86a7e732400a8e5f8c1fb0ae08f1"
format_stamp: "Formatted at 2026-08-10 06:36:30 on dist-test-slave-w206"
server_key: "50b192c101ff80be9f9b8c61d4695ce9"
server_key_iv: "9de826fdd777ae8cc73f8a3be4d40def"
server_key_version: "encryptionkey@0"
I20260810 06:36:30.928959 9071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:30.932296 9071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:30.932569 9071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:30.932713 9071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:30.932929 9071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:30.933233 9071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:30.933285 9071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.933326 9071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:30.933352 9071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:30.939137 9071 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:45541
I20260810 06:36:30.939172 9204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:45541 every 8 connection(s)
I20260810 06:36:30.939517 9071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:30.942704 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.940360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:37811 (local address 127.30.71.62:42089)
0810 06:36:30.940414 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:30.940417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:30.940685 (+ 268us) server_negotiation.cc:407] Connection header received
0810 06:36:30.941128 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:30.941129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:30.941135 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:30.941190 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:30.942484 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:30.942485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:30.942485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:30.942489 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:30.942502 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:30.942505 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:30.942557 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:30.942638 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 06:36:30.942651 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:36:30.942878 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:30.940358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42089 (local address 127.30.71.1:37811)
0810 06:36:30.940557 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:30.940590 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:30.941045 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:30.941306 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:30.941310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:30.941318 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:30.942346 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:36:30.942346 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:30.942373 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:36:30.942401 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:30.942568 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:30.942573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:30.942575 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:36:30.942633 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:36:30.942682 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260810 06:36:30.943475 9205 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42089
I20260810 06:36:30.943583 9205 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:30.943735 9205 heartbeater.cc:511] Master 127.30.71.62:42089 requested a full tablet report, sending...
I20260810 06:36:30.943822 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 9071
I20260810 06:36:30.943931 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:36:30.944154 31004 external_mini_cluster.cc:1428] Setting key 7a9bb8eb2bd5aa94b5b1a64bfe4376c3
I20260810 06:36:30.944312 9005 ts_manager.cc:194] Registered new tserver with Master: cc6e86a7e732400a8e5f8c1fb0ae08f1 (127.30.71.1:45541)
I20260810 06:36:30.945226 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42089
--builtin_ntp_servers=127.30.71.20:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:36:31.027102 9208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:31.027329 9208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:31.027354 9208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:31.028471 9208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:31.028505 9208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:31.028530 9208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:31.029816 9208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:31.030011 9208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:31.030335 9208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:31.035491 9217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:31.035606 9215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:31.035663 9214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:31.035602 9208 server_base.cc:1034] running on GCE node
I20260810 06:36:31.035974 9208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:31.036235 9208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:31.037406 9208 hybrid_clock.cc:648] HybridClock initialized: now 1786343791037392 us; error 44 us; skew 500 ppm
I20260810 06:36:31.038893 9208 webserver.cc:460] Webserver started at http://127.30.71.2:41017/ using document root <none> and password file <none>
I20260810 06:36:31.039168 9208 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:31.039242 9208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:31.039391 9208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:31.040484 9208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "856881be9cf64db3b2444374ba5c1165"
format_stamp: "Formatted at 2026-08-10 06:36:31 on dist-test-slave-w206"
server_key: "0845c4a70c79e6aa709797e4643f8a9e"
server_key_iv: "772a1ca0db8036a3c7018c01645a80ab"
server_key_version: "encryptionkey@0"
I20260810 06:36:31.040854 9208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "856881be9cf64db3b2444374ba5c1165"
format_stamp: "Formatted at 2026-08-10 06:36:31 on dist-test-slave-w206"
server_key: "0845c4a70c79e6aa709797e4643f8a9e"
server_key_iv: "772a1ca0db8036a3c7018c01645a80ab"
server_key_version: "encryptionkey@0"
I20260810 06:36:31.042181 9208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:31.043035 9224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:31.043217 9208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:31.043318 9208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "856881be9cf64db3b2444374ba5c1165"
format_stamp: "Formatted at 2026-08-10 06:36:31 on dist-test-slave-w206"
server_key: "0845c4a70c79e6aa709797e4643f8a9e"
server_key_iv: "772a1ca0db8036a3c7018c01645a80ab"
server_key_version: "encryptionkey@0"
I20260810 06:36:31.043406 9208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:31.046662 9208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:31.046929 9208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:31.047097 9208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:31.047308 9208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:31.047619 9208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:31.047672 9208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:31.047711 9208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:31.047739 9208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:31.053825 9208 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:34981
I20260810 06:36:31.053884 9345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:34981 every 8 connection(s)
I20260810 06:36:31.054193 9208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:36:31.057389 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:31.055021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:40553 (local address 127.30.71.62:42089)
0810 06:36:31.055074 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:31.055076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:31.055263 (+ 187us) server_negotiation.cc:407] Connection header received
0810 06:36:31.055722 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:31.055723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:31.055730 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:31.055811 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:31.057063 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:31.057065 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:31.057065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:31.057069 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:31.057086 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:31.057090 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:31.057144 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:31.057321 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 06:36:31.057335 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:36:31.057616 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:31.054939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42089 (local address 127.30.71.2:40553)
0810 06:36:31.055146 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:31.055184 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:31.055626 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:31.055862 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:31.055865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:31.055872 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:31.056891 (+ 1019us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:36:31.056891 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:31.056918 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:36:31.056946 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:31.057232 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:31.057239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:31.057241 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:36:31.057321 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:36:31.057370 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":117,"threads_started":1}
I20260810 06:36:31.058208 9346 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42089
I20260810 06:36:31.058298 9346 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:31.058418 9346 heartbeater.cc:511] Master 127.30.71.62:42089 requested a full tablet report, sending...
I20260810 06:36:31.058766 9004 ts_manager.cc:194] Registered new tserver with Master: 856881be9cf64db3b2444374ba5c1165 (127.30.71.2:34981)
I20260810 06:36:31.060215 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 9208
I20260810 06:36:31.060328 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:36:31.060557 31004 external_mini_cluster.cc:1428] Setting key 226fee8d2653cc805abdbdce4e15a0b4
I20260810 06:36:31.061564 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42089
--builtin_ntp_servers=127.30.71.20:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:36:31.143656 9349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:31.143872 9349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:31.143898 9349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:31.145007 9349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:31.145040 9349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:31.145064 9349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:31.146426 9349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:31.146626 9349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:31.146948 9349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:31.151976 9357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:31.151970 9354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:31.152148 9349 server_base.cc:1034] running on GCE node
W20260810 06:36:31.152034 9355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:31.152416 9349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:31.152673 9349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:31.153836 9349 hybrid_clock.cc:648] HybridClock initialized: now 1786343791153829 us; error 35 us; skew 500 ppm
I20260810 06:36:31.155259 9349 webserver.cc:460] Webserver started at http://127.30.71.3:33541/ using document root <none> and password file <none>
I20260810 06:36:31.155536 9349 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:31.155611 9349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:31.155764 9349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:31.156890 9349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "ab2e82089995463f8d2803e8b7840d6b"
format_stamp: "Formatted at 2026-08-10 06:36:31 on dist-test-slave-w206"
server_key: "7d6405f9183cdc33b5d323cfc778a70f"
server_key_iv: "67721256b355419fb2c4e3368e37fb0f"
server_key_version: "encryptionkey@0"
I20260810 06:36:31.157267 9349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "ab2e82089995463f8d2803e8b7840d6b"
format_stamp: "Formatted at 2026-08-10 06:36:31 on dist-test-slave-w206"
server_key: "7d6405f9183cdc33b5d323cfc778a70f"
server_key_iv: "67721256b355419fb2c4e3368e37fb0f"
server_key_version: "encryptionkey@0"
I20260810 06:36:31.158666 9349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:31.159555 9363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:31.159719 9349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:36:31.159826 9349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "ab2e82089995463f8d2803e8b7840d6b"
format_stamp: "Formatted at 2026-08-10 06:36:31 on dist-test-slave-w206"
server_key: "7d6405f9183cdc33b5d323cfc778a70f"
server_key_iv: "67721256b355419fb2c4e3368e37fb0f"
server_key_version: "encryptionkey@0"
I20260810 06:36:31.159924 9349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:31.163533 9349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:31.163815 9349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:31.163962 9349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:31.164173 9349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:31.164497 9349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:31.164553 9349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:31.164594 9349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:31.164623 9349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:31.170416 9349 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:33461
I20260810 06:36:31.170455 9487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:33461 every 8 connection(s)
I20260810 06:36:31.170933 9349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:36:31.174136 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:31.171743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:38001 (local address 127.30.71.62:42089)
0810 06:36:31.171802 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:31.171803 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:31.172068 (+ 265us) server_negotiation.cc:407] Connection header received
0810 06:36:31.172610 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:31.172612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:31.172619 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:31.172667 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:31.173830 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:31.173832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:31.173833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:36:31.173837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:31.173853 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:36:31.173858 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:31.173945 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:31.174041 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 06:36:31.174060 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:36:31.174364 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:31.171713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42089 (local address 127.30.71.3:38001)
0810 06:36:31.171946 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:31.171980 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:31.172486 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:31.172753 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:31.172757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:31.172765 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:31.173672 (+ 907us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:36:31.173672 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:31.173702 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:36:31.173729 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:31.173943 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:31.173950 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:31.173954 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:36:31.174052 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:36:31.174116 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260810 06:36:31.175036 9488 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42089
I20260810 06:36:31.175132 9488 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:31.175246 9488 heartbeater.cc:511] Master 127.30.71.62:42089 requested a full tablet report, sending...
I20260810 06:36:31.175613 9004 ts_manager.cc:194] Registered new tserver with Master: ab2e82089995463f8d2803e8b7840d6b (127.30.71.3:33461)
I20260810 06:36:31.176926 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 9349
I20260810 06:36:31.177009 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:36:31.177160 31004 external_mini_cluster.cc:1428] Setting key 574e2fd33216f6199ff909e5ed528d25
I20260810 06:36:31.178063 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:36:31.179884 9068 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:36:31.179442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.30.71.62:42089)
0810 06:36:31.179484 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:31.179486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:31.179588 (+ 102us) server_negotiation.cc:407] Connection header received
0810 06:36:31.179627 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:31.179627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:31.179632 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:31.179683 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:31.179800 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51846: BlockingRecv error: recv got EOF from 127.0.0.1:51846 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:36:31.180007 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 9071
I20260810 06:36:31.187470 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 9208
I20260810 06:36:31.197465 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 9349
I20260810 06:36:31.202814 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 8974
2026-08-10T06:36:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (540 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:36:31 dist-test-slave-w206 krb5kdc[9498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:36:31 dist-test-slave-w206 krb5kdc[9498](info): set up 2 sockets
Aug 10 06:36:31 dist-test-slave-w206 krb5kdc[9498](info): commencing operation
krb5kdc: starting...
W20260810 06:36:32.291996 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:36:32 dist-test-slave-w206 krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:36:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:36:32.419982 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33347
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:33347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:32.501747 9512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:32.501988 9512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:32.502024 9512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:32.503185 9512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:36:32.503249 9512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:32.503283 9512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:32.503315 9512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:36:32.503340 9512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:36:32.505283 9512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:33347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:32.505554 9512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:32.505787 9512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:32.510668 9519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:32.510761 9522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:32.510845 9518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:32.511135 9512 server_base.cc:1034] running on GCE node
I20260810 06:36:32.511402 9512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:32.511663 9512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:32.512817 9512 hybrid_clock.cc:648] HybridClock initialized: now 1786343792512802 us; error 31 us; skew 500 ppm
Aug 10 06:36:32 dist-test-slave-w206 krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343792, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:32.515460 9512 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:36:32.515939 9512 webserver.cc:460] Webserver started at http://127.30.71.62:39821/ using document root <none> and password file <none>
I20260810 06:36:32.516201 9512 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:32.516270 9512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:32.516433 9512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:32.517517 9512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "3dad9fc72ed6dea3df8d23bbccabb537"
server_key_iv: "0c76779627b5971287f395effac7dced"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.517886 9512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "3dad9fc72ed6dea3df8d23bbccabb537"
server_key_iv: "0c76779627b5971287f395effac7dced"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.519280 9512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 06:36:32.520054 9529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.520228 9512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:36:32.520341 9512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "3dad9fc72ed6dea3df8d23bbccabb537"
server_key_iv: "0c76779627b5971287f395effac7dced"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.520417 9512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:32.539776 9512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:32.541692 9512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:32.541868 9512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:32.545990 9512 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:33347
I20260810 06:36:32.546030 9588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:33347 every 8 connection(s)
I20260810 06:36:32.546397 9512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:36:32.547477 9589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:32.549757 9589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc: Bootstrap starting.
I20260810 06:36:32.550383 9589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:32.550686 9589 log.cc:824] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:32.551400 9589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc: No bootstrap required, opened a new log
I20260810 06:36:32.553015 9589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33347 } }
I20260810 06:36:32.553153 9589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:32.553176 9589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf0c2e97d1d41c589cf62a3d3e1cecc, State: Initialized, Role: FOLLOWER
I20260810 06:36:32.553331 9589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33347 } }
I20260810 06:36:32.553414 9589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:36:32.553438 9589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:36:32.553510 9589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:32.554337 9589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33347 } }
I20260810 06:36:32.554430 9589 leader_election.cc:302] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cf0c2e97d1d41c589cf62a3d3e1cecc; no voters:
I20260810 06:36:32.554644 9589 leader_election.cc:288] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:36:32.554701 9592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:32.554898 9592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [term 1 LEADER]: Becoming Leader. State: Replica: 7cf0c2e97d1d41c589cf62a3d3e1cecc, State: Running, Role: LEADER
I20260810 06:36:32.554993 9589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:36:32.555063 9592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33347 } }
I20260810 06:36:32.555399 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 9512
I20260810 06:36:32.555506 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:36:32.555641 9594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cf0c2e97d1d41c589cf62a3d3e1cecc. Latest consensus state: current_term: 1 leader_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33347 } } }
I20260810 06:36:32.555716 9594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:32.555757 31004 external_mini_cluster.cc:1428] Setting key 1787b5ed04fcf489f5a70991e6819f1d
I20260810 06:36:32.556048 9593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf0c2e97d1d41c589cf62a3d3e1cecc" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33347 } } }
I20260810 06:36:32.556159 9603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:36:32.556181 9593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc [sys.catalog]: This master's current role is: LEADER
I20260810 06:36:32.556780 9603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:36:32.558449 9603 catalog_manager.cc:1347] Generated new cluster ID: 00dd59f01172407598806e50b0c69775
I20260810 06:36:32.558501 9603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:36:32 dist-test-slave-w206 krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:32.568075 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:32.556474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.30.71.62:33347)
0810 06:36:32.556706 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:32.556716 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:32.556745 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:36:32.556867 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:32.556875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:32.556935 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:32.557131 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:32.557899 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.559279 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.560333 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.561006 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.561126 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:32.564142 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:32.564147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:32.564147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:32.564160 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:32.566146 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:32.566574 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:32.566577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:32.566577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:32.566713 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:32.566936 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:32.566937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:32.566937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:32.567284 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:32.567440 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:32.567568 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:36:32.567847 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
I20260810 06:36:32.591357 9603 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:36:32.592123 9603 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:36:32.599679 9603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc: Generated new TSK 0
I20260810 06:36:32.599905 9603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:36:32.633903 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33347
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:32.731647 9618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:32.731854 9618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:32.731877 9618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:32.733021 9618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:32.733067 9618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:32.733093 9618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:36:32.734553 9618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:32.734771 9618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:32.735029 9618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:32.739987 9626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:32.740018 9624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:32.740042 9623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:32.740111 9618 server_base.cc:1034] running on GCE node
I20260810 06:36:32.740404 9618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:32.740701 9618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:32.741874 9618 hybrid_clock.cc:648] HybridClock initialized: now 1786343792741866 us; error 43 us; skew 500 ppm
Aug 10 06:36:32 dist-test-slave-w206 krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343792, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:32.744544 9618 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:36:32.744928 9618 webserver.cc:460] Webserver started at http://127.30.71.1:40355/ using document root <none> and password file <none>
I20260810 06:36:32.745112 9618 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:32.745152 9618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:32.745240 9618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:32.746284 9618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "93cc963fe10a437bb1077466a8fee823"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "4888dd2765e23e79d9e6a61191da3ce3"
server_key_iv: "68f2874ec2c9ff52b0f977776fc8c060"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.746587 9618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "93cc963fe10a437bb1077466a8fee823"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "4888dd2765e23e79d9e6a61191da3ce3"
server_key_iv: "68f2874ec2c9ff52b0f977776fc8c060"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.747937 9618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:36:32.748610 9633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.748791 9618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:36:32.748868 9618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "93cc963fe10a437bb1077466a8fee823"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "4888dd2765e23e79d9e6a61191da3ce3"
server_key_iv: "68f2874ec2c9ff52b0f977776fc8c060"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.748925 9618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:32.757571 9618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:32.759500 9618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:32.759627 9618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:32.759807 9618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:32.760129 9618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:32.760177 9618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.760222 9618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:32.760249 9618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.766041 9618 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:39511
I20260810 06:36:32.766144 9756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:39511 every 8 connection(s)
I20260810 06:36:32.766459 9618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:36:32.769806 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 9618
I20260810 06:36:32.769889 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:36:32.770097 31004 external_mini_cluster.cc:1428] Setting key 62a2f70d4fc81453f3cc8c3bbbf016c9
Aug 10 06:36:32 dist-test-slave-w206 krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343792, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:32.776788 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:32.767346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:43127 (local address 127.30.71.62:33347)
0810 06:36:32.767393 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:32.767395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:32.767533 (+ 138us) server_negotiation.cc:407] Connection header received
0810 06:36:32.768058 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:32.768060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:32.768075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:32.768148 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:32.769094 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.769656 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.770950 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.771317 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.771428 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:32.773835 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:32.773837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:32.773837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:32.773845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:32.775366 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:32.775775 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:32.775777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:32.775777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:32.775837 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:32.776102 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:32.776103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:32.776103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:32.776219 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:32.776302 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:32.776590 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:36:32.776685 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:36:32.776949 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:32.767229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33347 (local address 127.30.71.1:43127)
0810 06:36:32.767405 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:32.767446 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:32.767938 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:32.768225 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:32.768228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:32.768798 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:32.769007 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:32.769012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.769754 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.769754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:32.770868 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:32.770871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.771446 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.771447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:32.771547 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:32.772102 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:32.772115 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:32.773716 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:32.775459 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:32.775465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:32.775467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:32.775695 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:32.775909 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:32.775910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:32.775910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:32.776026 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:32.776293 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:32.776299 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:32.776449 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 06:36:32.776553 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 06:36:32.776713 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260810 06:36:32.777506 9757 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33347
I20260810 06:36:32.777616 9757 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:32.777793 9757 heartbeater.cc:511] Master 127.30.71.62:33347 requested a full tablet report, sending...
I20260810 06:36:32.778347 9546 ts_manager.cc:194] Registered new tserver with Master: 93cc963fe10a437bb1077466a8fee823 (127.30.71.1:39511)
I20260810 06:36:32.778951 9546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:43127
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:36:32.797374 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33347
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:32.878748 9762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:32.878945 9762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:32.878986 9762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:32.880115 9762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:32.880154 9762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:32.880180 9762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:36:32.881599 9762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:32.881822 9762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:32.882041 9762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:32.887033 9767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:32.887154 9770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:32.887154 9768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:32.887238 9762 server_base.cc:1034] running on GCE node
I20260810 06:36:32.887583 9762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:32.887838 9762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:32.888984 9762 hybrid_clock.cc:648] HybridClock initialized: now 1786343792888973 us; error 32 us; skew 500 ppm
Aug 10 06:36:32 dist-test-slave-w206 krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343792, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:32.891433 9762 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:36:32.891892 9762 webserver.cc:460] Webserver started at http://127.30.71.2:37653/ using document root <none> and password file <none>
I20260810 06:36:32.892153 9762 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:32.892199 9762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:32.892288 9762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:32.893278 9762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "c85362cc9e5442f1b008f1308ab82363"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "8d79c9882f5e7d3e811f92f5dc1d0809"
server_key_iv: "b8dafc82f46b6157ee47392db1310c25"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.893573 9762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "c85362cc9e5442f1b008f1308ab82363"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "8d79c9882f5e7d3e811f92f5dc1d0809"
server_key_iv: "b8dafc82f46b6157ee47392db1310c25"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.894842 9762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:36:32.895610 9777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.895780 9762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.895895 9762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "c85362cc9e5442f1b008f1308ab82363"
format_stamp: "Formatted at 2026-08-10 06:36:32 on dist-test-slave-w206"
server_key: "8d79c9882f5e7d3e811f92f5dc1d0809"
server_key_iv: "b8dafc82f46b6157ee47392db1310c25"
server_key_version: "encryptionkey@0"
I20260810 06:36:32.895972 9762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:32.903473 9762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:32.905351 9762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:32.905509 9762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:32.905720 9762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:32.906018 9762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:32.906073 9762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.906112 9762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:32.906141 9762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:32.912398 9762 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:34839
I20260810 06:36:32.912436 9900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:34839 every 8 connection(s)
I20260810 06:36:32.912745 9762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:36:32 dist-test-slave-w206 krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343792, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:32.921586 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:32.913459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:33397 (local address 127.30.71.62:33347)
0810 06:36:32.913526 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:32.913528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:32.913747 (+ 219us) server_negotiation.cc:407] Connection header received
0810 06:36:32.914212 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:32.914213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:32.914222 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:32.914279 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:32.915201 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.915711 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.916710 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.916896 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.916949 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:32.919210 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:32.919212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:32.919212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:32.919219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:32.920448 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:32.920812 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:32.920813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:32.920814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:32.920866 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:32.921074 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:32.921075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:32.921075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:32.921172 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:32.921222 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:32.921452 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 06:36:32.921522 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:36:32.921772 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:32.913450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33347 (local address 127.30.71.2:33397)
0810 06:36:32.913616 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:32.913655 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:32.914130 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:32.914329 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:32.914332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:32.914876 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:32.915127 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:32.915132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.915850 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.915852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:32.916633 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:32.916636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:32.916971 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:32.916972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:32.917075 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:32.917611 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:32.917624 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:32.919122 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:32.920522 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:32.920528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:32.920530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:32.920737 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:32.920925 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:32.920925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:32.920926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:32.921021 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:32.921225 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:32.921228 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:32.921345 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 06:36:32.921425 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:36:32.921539 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260810 06:36:32.922315 9901 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33347
I20260810 06:36:32.922423 9901 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:32.922580 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 9762
I20260810 06:36:32.922647 9901 heartbeater.cc:511] Master 127.30.71.62:33347 requested a full tablet report, sending...
I20260810 06:36:32.922675 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:36:32.922899 31004 external_mini_cluster.cc:1428] Setting key a753e3a205745714ab35b8dff6372223
I20260810 06:36:32.923061 9546 ts_manager.cc:194] Registered new tserver with Master: c85362cc9e5442f1b008f1308ab82363 (127.30.71.2:34839)
I20260810 06:36:32.923727 9546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:33397
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:36:32.948038 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33347
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:36:33.029868 9907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:36:33.030119 9907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:36:33.030159 9907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:36:33.031447 9907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:36:33.031524 9907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:36:33.031572 9907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:36:33.033088 9907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:36:33.033372 9907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:36:33.033615 9907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:36:33.038785 9913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:36:33.038810 9912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:33.038918 9907 server_base.cc:1034] running on GCE node
W20260810 06:36:33.038815 9915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:36:33.039237 9907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:36:33.039491 9907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:36:33.040666 9907 hybrid_clock.cc:648] HybridClock initialized: now 1786343793040649 us; error 46 us; skew 500 ppm
Aug 10 06:36:33 dist-test-slave-w206 krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343793, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:36:33.043385 9907 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:36:33.043763 9907 webserver.cc:460] Webserver started at http://127.30.71.3:39943/ using document root <none> and password file <none>
I20260810 06:36:33.043953 9907 fs_manager.cc:359] Metadata directory not provided
I20260810 06:36:33.043996 9907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:36:33.044087 9907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:36:33.045085 9907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "7fa4c2b42645432688774403ba042d2d"
format_stamp: "Formatted at 2026-08-10 06:36:33 on dist-test-slave-w206"
server_key: "1856021cc5ce6d17ce9de5a67ea4094c"
server_key_iv: "059473240374d7aa0c4b66b9b09e89cd"
server_key_version: "encryptionkey@0"
I20260810 06:36:33.045389 9907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "7fa4c2b42645432688774403ba042d2d"
format_stamp: "Formatted at 2026-08-10 06:36:33 on dist-test-slave-w206"
server_key: "1856021cc5ce6d17ce9de5a67ea4094c"
server_key_iv: "059473240374d7aa0c4b66b9b09e89cd"
server_key_version: "encryptionkey@0"
I20260810 06:36:33.046628 9907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:36:33.047427 9922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:33.047603 9907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:36:33.047712 9907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "7fa4c2b42645432688774403ba042d2d"
format_stamp: "Formatted at 2026-08-10 06:36:33 on dist-test-slave-w206"
server_key: "1856021cc5ce6d17ce9de5a67ea4094c"
server_key_iv: "059473240374d7aa0c4b66b9b09e89cd"
server_key_version: "encryptionkey@0"
I20260810 06:36:33.047771 9907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:36:33.064000 9907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:36:33.065863 9907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:36:33.065974 9907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:36:33.066151 9907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:36:33.066480 9907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:36:33.066519 9907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:33.066565 9907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:36:33.066591 9907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:36:33.073726 9907 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:41001
I20260810 06:36:33.073788 10045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:41001 every 8 connection(s)
I20260810 06:36:33.074097 9907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:36:33 dist-test-slave-w206 krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343793, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:36:33.082883 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 9907
I20260810 06:36:33.083009 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:36:33.083250 31004 external_mini_cluster.cc:1428] Setting key 327c2836efe4473de4b7cf8c548e2366
I20260810 06:36:33.084056 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.075006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:58557 (local address 127.30.71.62:33347)
0810 06:36:33.075080 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.075082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.075262 (+ 180us) server_negotiation.cc:407] Connection header received
0810 06:36:33.075834 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.075835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.075849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.075902 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:33.076873 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.077389 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.078432 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.078663 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.078727 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.080991 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:33.080992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:33.080992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:33.081000 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:33.082308 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:33.082820 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:33.082822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:33.082822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:33.082891 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:33.083249 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:33.083251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:33.083251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:33.083431 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:33.083503 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.083896 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.083985 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:36:33.084281 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.074885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33347 (local address 127.30.71.3:58557)
0810 06:36:33.075114 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:33.075180 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:33.075713 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:33.075988 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:33.075991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:33.076572 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:36:33.076764 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.076769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.077493 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.077494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.078322 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.078324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.078747 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.078747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.078868 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.079412 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:36:33.079421 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:36:33.080902 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:36:33.082387 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:33.082393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:33.082395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:33.082632 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:33.083004 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:36:33.083005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:36:33.083005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:36:33.083165 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:36:33.083513 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:36:33.083520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:36:33.083705 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 06:36:33.083832 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 06:36:33.084014 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":132,"threads_started":1}
I20260810 06:36:33.084681 10046 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33347
I20260810 06:36:33.084770 10046 heartbeater.cc:464] Registering TS with master...
I20260810 06:36:33.084910 10046 heartbeater.cc:511] Master 127.30.71.62:33347 requested a full tablet report, sending...
I20260810 06:36:33.085212 9546 ts_manager.cc:194] Registered new tserver with Master: 7fa4c2b42645432688774403ba042d2d (127.30.71.3:41001)
I20260810 06:36:33.085536 9546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:58557
I20260810 06:36:33.095235 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:36:33.102519 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.096506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35196 (local address 127.30.71.62:33347)
0810 06:36:33.096583 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.096585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.096699 (+ 114us) server_negotiation.cc:407] Connection header received
0810 06:36:33.096794 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.096795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.096805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.096872 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:36:33.097432 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.097913 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.098733 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.098926 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.099044 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.100298 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:36:33.100299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:36:33.100300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:36:33.100306 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:36:33.101523 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:33.101838 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:33.101839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:33.101839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:33.101891 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:36:33.102050 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:36:33.102051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:36:33.102051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:36:33.102184 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:36:33.102260 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.102343 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.102413 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:36:33.104153 9546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:36:33.105706 9546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:36:33.112377 10059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.108232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:39511 (local address 127.0.0.1:34796)
0810 06:36:33.108485 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:33.108495 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:33.108538 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:33.108926 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:33.108927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:33.108928 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:33.109001 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.109003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.109901 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.109902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.111085 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.111088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.112013 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.112014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.112107 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.112123 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 06:36:33.112192 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:36:33.112281 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260810 06:36:33.112380 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.108163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:34839 (local address 127.0.0.1:46928)
0810 06:36:33.108317 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:33.108389 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:33.108458 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:33.108917 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:33.108920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:33.108923 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:33.109005 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.109006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.110369 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.110370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.111367 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.111370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.112111 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.112112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.112183 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.112187 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:36:33.112248 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:36:33.112294 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":2,"thread_start_us":51,"threads_started":1}
I20260810 06:36:33.112722 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.108505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.30.71.1:39511)
0810 06:36:33.108672 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.108676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.108689 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:36:33.108731 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.108733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.108750 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.108842 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:33.109071 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.109788 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.111173 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.111910 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.111991 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.112049 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.112593 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.112640 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260810 06:36:33.113267 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.108274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46928 (local address 127.30.71.2:34839)
0810 06:36:33.108742 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.108745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.108754 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:33.108791 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.108793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.108807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.109082 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:33.109158 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.110253 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.111467 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.112036 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.112110 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.112167 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.113110 (+ 943us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.113178 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":103,"threads_started":1}
I20260810 06:36:33.113547 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.108481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34830 (local address 127.30.71.3:41001)
0810 06:36:33.108650 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.108655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.108666 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:36:33.108774 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.108777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.108793 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.108870 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:33.109115 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.110253 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.112271 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.112975 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.113035 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.113070 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.113427 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.113468 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260810 06:36:33.113303 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.108263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:41001 (local address 127.0.0.1:34830)
0810 06:36:33.108611 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:33.108621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:33.108656 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:33.108963 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:33.108964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:33.108966 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:33.109047 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.109049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.111051 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.111052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.112194 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.112196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.113057 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.113058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.113116 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.113121 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:36:33.113180 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:36:33.113226 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":23,"thread_start_us":63,"threads_started":1}
I20260810 06:36:33.113658 9684 tablet_service.cc:1467] Processing CreateTablet for tablet d5fbabba6c2e4d4294b79fc7088939ea (DEFAULT_TABLE table=test-table [id=209a164724224c9392c9549308b72778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:33.113859 9828 tablet_service.cc:1467] Processing CreateTablet for tablet d5fbabba6c2e4d4294b79fc7088939ea (DEFAULT_TABLE table=test-table [id=209a164724224c9392c9549308b72778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:33.114043 9684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fbabba6c2e4d4294b79fc7088939ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:33.114162 9828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fbabba6c2e4d4294b79fc7088939ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:33.114342 9975 tablet_service.cc:1467] Processing CreateTablet for tablet d5fbabba6c2e4d4294b79fc7088939ea (DEFAULT_TABLE table=test-table [id=209a164724224c9392c9549308b72778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:36:33.114583 9975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fbabba6c2e4d4294b79fc7088939ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:36:33.117226 10065 tablet_bootstrap.cc:492] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Bootstrap starting.
I20260810 06:36:33.117671 10067 tablet_bootstrap.cc:492] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823: Bootstrap starting.
I20260810 06:36:33.117659 10066 tablet_bootstrap.cc:492] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Bootstrap starting.
I20260810 06:36:33.117977 10065 tablet_bootstrap.cc:654] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:33.118326 10066 tablet_bootstrap.cc:654] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:33.118326 10067 tablet_bootstrap.cc:654] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823: Neither blocks nor log segments found. Creating new log.
I20260810 06:36:33.118487 10065 log.cc:824] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:33.118947 10067 log.cc:824] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:33.119015 10066 log.cc:824] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Log is configured to *not* fsync() on all Append() calls
I20260810 06:36:33.119328 10065 tablet_bootstrap.cc:492] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: No bootstrap required, opened a new log
I20260810 06:36:33.119413 10065 ts_tablet_manager.cc:1397] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:36:33.119717 10067 tablet_bootstrap.cc:492] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823: No bootstrap required, opened a new log
I20260810 06:36:33.119801 10067 ts_tablet_manager.cc:1397] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:36:33.121002 10065 raft_consensus.cc:348] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.121187 10065 raft_consensus.cc:374] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:33.121241 10065 raft_consensus.cc:729] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa4c2b42645432688774403ba042d2d, State: Initialized, Role: FOLLOWER
I20260810 06:36:33.121363 10065 consensus_queue.cc:260] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.121408 10067 raft_consensus.cc:348] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.121573 10067 raft_consensus.cc:374] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:33.121618 10067 raft_consensus.cc:729] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93cc963fe10a437bb1077466a8fee823, State: Initialized, Role: FOLLOWER
I20260810 06:36:33.121704 10046 heartbeater.cc:503] Master 127.30.71.62:33347 was elected leader, sending a full tablet report...
I20260810 06:36:33.121737 10067 consensus_queue.cc:260] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.121695 10065 ts_tablet_manager.cc:1428] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:36:33.122074 10067 ts_tablet_manager.cc:1428] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:36:33.122442 9757 heartbeater.cc:503] Master 127.30.71.62:33347 was elected leader, sending a full tablet report...
I20260810 06:36:33.122632 10066 tablet_bootstrap.cc:492] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: No bootstrap required, opened a new log
I20260810 06:36:33.122702 10066 ts_tablet_manager.cc:1397] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:36:33.124266 10066 raft_consensus.cc:348] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.124393 10066 raft_consensus.cc:374] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:36:33.124415 10066 raft_consensus.cc:729] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85362cc9e5442f1b008f1308ab82363, State: Initialized, Role: FOLLOWER
I20260810 06:36:33.124555 10066 consensus_queue.cc:260] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.124830 10066 ts_tablet_manager.cc:1428] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:36:33.124848 9901 heartbeater.cc:503] Master 127.30.71.62:33347 was elected leader, sending a full tablet report...
W20260810 06:36:33.163349 9902 tablet.cc:2369] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:36:33.179946 10071 raft_consensus.cc:482] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:36:33.180092 10071 raft_consensus.cc:504] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.180452 10071 leader_election.cc:288] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c85362cc9e5442f1b008f1308ab82363 (127.30.71.2:34839), 93cc963fe10a437bb1077466a8fee823 (127.30.71.1:39511)
I20260810 06:36:33.183687 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.180857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:52719 (local address 127.30.71.1:39511)
0810 06:36:33.180897 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.180901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.180914 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:36:33.180940 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.180941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.180951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.181028 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:33.181190 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.181688 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.182886 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.183324 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.183409 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.183420 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.183577 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.183615 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:36:33.183696 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.180797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:41769 (local address 127.30.71.2:34839)
0810 06:36:33.180850 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.180853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.181040 (+ 187us) server_negotiation.cc:407] Connection header received
0810 06:36:33.181092 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.181093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.181106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.181177 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:36:33.181348 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.181799 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.182926 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.183337 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.183426 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.183436 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.183592 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.183630 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:36:33.183687 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.180679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:39511 (local address 127.30.71.3:52719)
0810 06:36:33.180746 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:33.180759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:33.180827 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:33.181034 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:33.181035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:33.181037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:33.181118 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.181119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.181782 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.181783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.182801 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.182804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.183407 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.183408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.183486 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.183489 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:36:33.183559 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:36:33.183600 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":3}
I20260810 06:36:33.183710 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.180694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:34839 (local address 127.30.71.3:41769)
0810 06:36:33.180957 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:33.180966 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:36:33.181040 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:36:33.181180 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:36:33.181181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:36:33.181182 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:36:33.181266 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.181268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.181882 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.181883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.182838 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:36:33.182840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.183427 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.183428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:36:33.183499 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.183503 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:36:33.183573 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:36:33.183612 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":73,"thread_start_us":140,"threads_started":1}
I20260810 06:36:33.184078 9705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fbabba6c2e4d4294b79fc7088939ea" candidate_uuid: "7fa4c2b42645432688774403ba042d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93cc963fe10a437bb1077466a8fee823" is_pre_election: true
I20260810 06:36:33.184108 9848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fbabba6c2e4d4294b79fc7088939ea" candidate_uuid: "7fa4c2b42645432688774403ba042d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85362cc9e5442f1b008f1308ab82363" is_pre_election: true
I20260810 06:36:33.184253 9705 raft_consensus.cc:2454] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa4c2b42645432688774403ba042d2d in term 0.
I20260810 06:36:33.184278 9848 raft_consensus.cc:2454] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa4c2b42645432688774403ba042d2d in term 0.
I20260810 06:36:33.184494 9925 leader_election.cc:302] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fa4c2b42645432688774403ba042d2d, 93cc963fe10a437bb1077466a8fee823; no voters:
I20260810 06:36:33.184634 10071 raft_consensus.cc:2781] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:36:33.184692 10071 raft_consensus.cc:482] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:36:33.184722 10071 raft_consensus.cc:3037] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:33.185393 10071 raft_consensus.cc:504] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.185529 10071 leader_election.cc:288] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [CANDIDATE]: Term 1 election: Requested vote from peers c85362cc9e5442f1b008f1308ab82363 (127.30.71.2:34839), 93cc963fe10a437bb1077466a8fee823 (127.30.71.1:39511)
I20260810 06:36:33.185667 9848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fbabba6c2e4d4294b79fc7088939ea" candidate_uuid: "7fa4c2b42645432688774403ba042d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85362cc9e5442f1b008f1308ab82363"
I20260810 06:36:33.185755 9848 raft_consensus.cc:3037] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:33.185737 9705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fbabba6c2e4d4294b79fc7088939ea" candidate_uuid: "7fa4c2b42645432688774403ba042d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93cc963fe10a437bb1077466a8fee823"
I20260810 06:36:33.185827 9705 raft_consensus.cc:3037] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:36:33.186537 9705 raft_consensus.cc:2454] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fa4c2b42645432688774403ba042d2d in term 1.
I20260810 06:36:33.186560 9848 raft_consensus.cc:2454] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fa4c2b42645432688774403ba042d2d in term 1.
I20260810 06:36:33.186715 9925 leader_election.cc:302] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fa4c2b42645432688774403ba042d2d, 93cc963fe10a437bb1077466a8fee823; no voters:
I20260810 06:36:33.186852 10071 raft_consensus.cc:2781] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:36:33.187048 10071 raft_consensus.cc:686] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 1 LEADER]: Becoming Leader. State: Replica: 7fa4c2b42645432688774403ba042d2d, State: Running, Role: LEADER
I20260810 06:36:33.187160 10071 consensus_queue.cc:237] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } }
I20260810 06:36:33.187999 9546 catalog_manager.cc:5495] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa4c2b42645432688774403ba042d2d (127.30.71.3). New cstate: current_term: 1 leader_uuid: "7fa4c2b42645432688774403ba042d2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fa4c2b42645432688774403ba042d2d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 41001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 } health_report { overall_health: UNKNOWN } } }
I20260810 06:36:33.205425 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.202912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.30.71.3:41001)
0810 06:36:33.202967 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:36:33.202987 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 06:36:33.202996 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:36:33.203063 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:36:33.203064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:36:33.203084 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:36:33.203170 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:36:33.203357 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.203830 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.204599 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:36:33.204811 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:36:33.204897 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:36:33.205067 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:36:33.205170 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:36:33.205255 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:36:33.205309 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 06:36:33.205328 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 06:36:33.207263 9705 raft_consensus.cc:1261] T d5fbabba6c2e4d4294b79fc7088939ea P 93cc963fe10a437bb1077466a8fee823 [term 1 FOLLOWER]: Refusing update from remote peer 7fa4c2b42645432688774403ba042d2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:33.207263 9848 raft_consensus.cc:1261] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 1 FOLLOWER]: Refusing update from remote peer 7fa4c2b42645432688774403ba042d2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:36:33.207608 10071 consensus_queue.cc:1035] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c85362cc9e5442f1b008f1308ab82363" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 34839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:33.207749 10075 consensus_queue.cc:1035] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "93cc963fe10a437bb1077466a8fee823" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 39511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:36:33.210196 10078 mvcc.cc:205] Tried to move back new op lower bound from 7316864176974036992 to 7316864176895262720. Current Snapshot: MvccSnapshot[applied={T|T < 7316864176974036992}]
I20260810 06:36:33.211232 10080 mvcc.cc:205] Tried to move back new op lower bound from 7316864176974036992 to 7316864176895262720. Current Snapshot: MvccSnapshot[applied={T|T < 7316864176974036992}]
I20260810 06:36:33.213052 9545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:33.213158 9545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:36:33.213971 9545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cf0c2e97d1d41c589cf62a3d3e1cecc: Sending DeleteTablet for 3 replicas of tablet d5fbabba6c2e4d4294b79fc7088939ea
I20260810 06:36:33.214334 9828 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fbabba6c2e4d4294b79fc7088939ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:33 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:46928
I20260810 06:36:33.214349 9975 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fbabba6c2e4d4294b79fc7088939ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:33 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:34830
I20260810 06:36:33.214354 9684 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fbabba6c2e4d4294b79fc7088939ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:36:33 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:34796
I20260810 06:36:33.214622 10090 tablet_replica.cc:330] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: stopping tablet replica
I20260810 06:36:33.214730 10090 raft_consensus.cc:2229] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:36:33.214854 10090 raft_consensus.cc:2258] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:33.214924 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 9618
I20260810 06:36:33.216305 10090 ts_tablet_manager.cc:1905] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:33.217582 10089 tablet_replica.cc:330] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: stopping tablet replica
I20260810 06:36:33.217715 10089 raft_consensus.cc:2229] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:36:33.217869 10089 raft_consensus.cc:2258] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:36:33.218235 10090 ts_tablet_manager.cc:1918] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:33.218369 10090 log.cc:1196] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/d5fbabba6c2e4d4294b79fc7088939ea
I20260810 06:36:33.218393 10089 ts_tablet_manager.cc:1905] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:36:33.218770 10090 ts_tablet_manager.cc:1939] T d5fbabba6c2e4d4294b79fc7088939ea P c85362cc9e5442f1b008f1308ab82363: Deleting consensus metadata
I20260810 06:36:33.219300 9530 catalog_manager.cc:4841] TS c85362cc9e5442f1b008f1308ab82363 (127.30.71.2:34839): tablet d5fbabba6c2e4d4294b79fc7088939ea (table test-table [id=209a164724224c9392c9549308b72778]) successfully deleted
I20260810 06:36:33.219915 10089 ts_tablet_manager.cc:1918] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:36:33.220005 10089 log.cc:1196] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/d5fbabba6c2e4d4294b79fc7088939ea
I20260810 06:36:33.220247 10089 ts_tablet_manager.cc:1939] T d5fbabba6c2e4d4294b79fc7088939ea P 7fa4c2b42645432688774403ba042d2d: Deleting consensus metadata
I20260810 06:36:33.223304 9533 catalog_manager.cc:4841] TS 7fa4c2b42645432688774403ba042d2d (127.30.71.3:41001): tablet d5fbabba6c2e4d4294b79fc7088939ea (table test-table [id=209a164724224c9392c9549308b72778]) successfully deleted
W20260810 06:36:33.226954 9532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:39511 (error 108)
I20260810 06:36:33.227192 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 9762
I20260810 06:36:33.228205 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:36:33.227339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:39511 (local address 127.0.0.1:34808)
0810 06:36:33.228144 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0810 06:36:33.228177 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:39511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":789}
W20260810 06:36:33.228909 9532 catalog_manager.cc:4584] TS 93cc963fe10a437bb1077466a8fee823 (127.30.71.1:39511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5fbabba6c2e4d4294b79fc7088939ea: Network error: Client connection negotiation failed: client connection to 127.30.71.1:39511: connect: Connection refused (error 111)
I20260810 06:36:33.232824 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 9907
I20260810 06:36:33.238886 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 9512
2026-08-10T06:36:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2037 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:36:33 dist-test-slave-w206 krb5kdc[10095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:36:33 dist-test-slave-w206 krb5kdc[10095](info): set up 2 sockets
Aug 10 06:36:33 dist-test-slave-w206 krb5kdc[10095](info): commencing operation
krb5kdc: starting...
W20260810 06:36:34.323561 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:36:34 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:36:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/rangeradmin_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/rangeradmin_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/rangerlookup_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/rangerlookup_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
I20260810 06:36:34.496101 31004 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskI3NzPm/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/postgres -l logfile start
2026-08-10 06:36:37.119 UTC [10127] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:36:37.120 UTC [10127] LOG: listening on IPv4 address "127.30.71.20", port 38633
2026-08-10 06:36:37.124 UTC [10127] LOG: listening on Unix socket "/tmp/.s.PGSQL.38633"
2026-08-10 06:36:37.146 UTC [10130] LOG: database system was shut down at 2026-08-10 06:36:35 UTC
2026-08-10 06:36:37.149 UTC [10127] LOG: database system is ready to accept connections
I20260810 06:36:38.188985 31004 mini_postgres.cc:96] Postgres bound to 38633
2026-08-10 06:36:38.196 UTC [10138] FATAL: database "slave" does not exist
127.30.71.20:38633 - accepting connections
I20260810 06:36:38.197350 31004 mini_ranger.cc:161] Starting Ranger...
I20260810 06:36:38.213581 31004 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:36:38.708463 31004 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:36:38.708535 31004 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-admin
2026-08-10 06:36:39,098 [I] DB FLAVOR :POSTGRES
2026-08-10 06:36:39,098 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:36:39,098 [I] Checking connection..
2026-08-10 06:36:39,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:36:39,382 [I] Checking connection passed.
2026-08-10 06:36:39,382 [I] --------- Verifying version history table ---------
2026-08-10 06:36:39,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:36:39,625 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:36:39,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:36:39,908 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:36:39,908 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:36:39,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:36:40.161 UTC [10207] WARNING: there is no transaction in progress
2026-08-10 06:36:40,169 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:36:40,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:36:40,421 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:36:40,421 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:36:40,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:36:40,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:36:40,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w206.c.gcp-upstream.internal','N') ;"
2026-08-10 06:36:41,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:36:41,445 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:36:41,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:36:41,708 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:36:41,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:36:41,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:36:42,251 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:36:42,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:36:43.298 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.316 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.334 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.351 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.370 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.391 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.463 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.470 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.487 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.498 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.509 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.524 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.535 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.542 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.550 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:43.557 UTC [10325] WARNING: there is no transaction in progress
2026-08-10 06:36:44,269 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:36:44,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:36:44,531 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:36:44,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:36:44,794 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:36:44,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:36:45,037 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:36:45,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:36:45,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:36:45,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w206.c.gcp-upstream.internal';"
2026-08-10 06:36:45,818 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:36:45,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:36:46,088 [I] DB_PATCHES have already been applied
I20260810 06:36:46.093827 31004 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-admin:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskI3NzPm/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:36:46.093904 31004 mini_ranger.cc:193] Using host: 127.30.71.20
I20260810 06:36:46.097607 31004 mini_ranger.cc:227] Ranger admin URL: http://127.30.71.20:48185
Aug 10, 2026 6:36:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:36:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:36:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:36:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:36:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:36:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:36:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:36:47.332950 31004 mini_ranger.cc:160] Time spent starting Ranger: real 9.136s user 0.002s sys 0.008s
Aug 10, 2026 6:37:09 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:37:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:37:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:37:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:37:10 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:37:11.011940 31004 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/rangerkms_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/rangerkms_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.71.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.71.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:37:11 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343831, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:37:11.101930 31004 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:37:11.112715 31004 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:37:12.029870 31004 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:37:12.029949 31004 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms
I20260810 06:37:12.029963 31004 mini_ranger_kms.cc:227] Using postgres at 127.30.71.20:38633
2026-08-10 06:37:12,582 [I] DB FLAVOR :POSTGRES
2026-08-10 06:37:12,582 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:37:12,583 [I] Checking connection
2026-08-10 06:37:12,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:37:13,151 [I] connection success
2026-08-10 06:37:13,152 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:37:13,152 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:37:13,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:37:13,581 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:37:13,581 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:37:13,581 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:37:13,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:38633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:37:13,946 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:37:14.451763 31004 mini_ranger_kms.cc:310] Created kms service
I20260810 06:37:14.677245 31004 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:37:14.724253 31004 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:37:14 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343831, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:37:15.155109 31004 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:37:15.155225 31004 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms:/tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:37:15.155251 31004 mini_ranger_kms.cc:240] Using host: 127.30.71.20
I20260810 06:37:15.159555 31004 mini_ranger_kms.cc:276] Ranger KMS PID: 10522
I20260810 06:37:15.159689 31004 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.71.20:45915
Aug 10, 2026 6:37:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:37:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:37:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:37:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:37:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:37:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 06:37:17.551947 31004 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.450s user 0.016s sys 0.482s
I20260810 06:37:17.552047 31004 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:37:17.552107 31004 mini_ranger_kms.cc:401] 127.30.71.20:45915/kms/v1/keys
Aug 10 06:37:17 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786343837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.71.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 06:37:18 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.71.20@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
Aug 10 06:37:18 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.71.20@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
Aug 10 06:37:21 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:37:21.074573 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:35309
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:35309
--ranger_config_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:45915/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:21.202739 10575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:21.202955 10575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:21.203037 10575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:21.204326 10575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:37:21.204372 10575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:21.204387 10575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:21.204399 10575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:37:21.204412 10575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:37:21.206041 10575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.71.20:45915/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:35309
--ranger_config_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:35309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:37:21.206308 10575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:21.206568 10575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:21.212044 10581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:21.212072 10580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:21.212246 10583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:21.212567 10575 server_base.cc:1034] running on GCE node
I20260810 06:37:21.212821 10575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:21.213161 10575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:21.214365 10575 hybrid_clock.cc:648] HybridClock initialized: now 1786343841214341 us; error 41 us; skew 500 ppm
Aug 10 06:37:21 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343841, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:21.222812 10575 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:37:21.223420 10575 webserver.cc:460] Webserver started at http://127.30.71.62:35187/ using document root <none> and password file <none>
I20260810 06:37:21.223681 10575 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:21.223724 10575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:21.223878 10575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:37:21 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343841, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:37:21.288856 10575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "3c183dcc63384583857f1b36dfa13c2e"
format_stamp: "Formatted at 2026-08-10 06:37:21 on dist-test-slave-w206"
server_key: "19b584bd83577aa18a9f1dd420316130"
server_key_iv: "d09e8c7c50b90bc6a8ed6c45dce00c3c"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:21.289359 10575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "3c183dcc63384583857f1b36dfa13c2e"
format_stamp: "Formatted at 2026-08-10 06:37:21 on dist-test-slave-w206"
server_key: "19b584bd83577aa18a9f1dd420316130"
server_key_iv: "d09e8c7c50b90bc6a8ed6c45dce00c3c"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:21.291325 10575 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:37:21.307132 10593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:21.307412 10575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:37:21.307544 10575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "3c183dcc63384583857f1b36dfa13c2e"
format_stamp: "Formatted at 2026-08-10 06:37:21 on dist-test-slave-w206"
server_key: "19b584bd83577aa18a9f1dd420316130"
server_key_iv: "d09e8c7c50b90bc6a8ed6c45dce00c3c"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:21.307662 10575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:21.325376 10575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:21.327739 10575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:21.328006 10575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:21.336517 10575 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:35309
I20260810 06:37:21.336566 10652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:35309 every 8 connection(s)
I20260810 06:37:21.336958 10575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:37:21.337754 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:21.340868 10653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e: Bootstrap starting.
I20260810 06:37:21.341825 10653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:21.342525 10653 log.cc:824] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:21.343356 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 10575
I20260810 06:37:21.343475 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:37:21.343482 10653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e: No bootstrap required, opened a new log
I20260810 06:37:21.343683 31004 external_mini_cluster.cc:1428] Setting key 339fae97a97d508ba0b537fe0a1b4b1a
I20260810 06:37:21.347393 10653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c183dcc63384583857f1b36dfa13c2e" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35309 } }
I20260810 06:37:21.347577 10653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:21.347649 10653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c183dcc63384583857f1b36dfa13c2e, State: Initialized, Role: FOLLOWER
I20260810 06:37:21.347842 10653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c183dcc63384583857f1b36dfa13c2e" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35309 } }
I20260810 06:37:21.347951 10653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:37:21.347988 10653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:37:21.348075 10653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:21.348984 10653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c183dcc63384583857f1b36dfa13c2e" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35309 } }
I20260810 06:37:21.349120 10653 leader_election.cc:302] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c183dcc63384583857f1b36dfa13c2e; no voters:
I20260810 06:37:21.349329 10653 leader_election.cc:288] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:37:21.349598 10653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:37:21.350317 10659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:21.350700 10659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [term 1 LEADER]: Becoming Leader. State: Replica: 3c183dcc63384583857f1b36dfa13c2e, State: Running, Role: LEADER
I20260810 06:37:21.351394 10659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c183dcc63384583857f1b36dfa13c2e" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35309 } }
I20260810 06:37:21.352716 10665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c183dcc63384583857f1b36dfa13c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c183dcc63384583857f1b36dfa13c2e" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35309 } } }
I20260810 06:37:21.352830 10665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:21.352875 10658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c183dcc63384583857f1b36dfa13c2e. Latest consensus state: current_term: 1 leader_uuid: "3c183dcc63384583857f1b36dfa13c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c183dcc63384583857f1b36dfa13c2e" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 35309 } } }
I20260810 06:37:21.352936 10658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e [sys.catalog]: This master's current role is: LEADER
I20260810 06:37:21.353296 10673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:37:21.353713 10653 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 10 06:37:21 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:37:21.359182 10673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:37:21.364732 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:21.344452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.30.71.62:35309)
0810 06:37:21.344718 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:21.344727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:21.344788 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:37:21.347440 (+ 2652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:21.347451 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:21.347525 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:21.347683 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:21.348705 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:21.350174 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:21.351540 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:21.352315 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:21.352429 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:21.357881 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:21.358008 (+ 127us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:21.358008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:21.358141 (+ 133us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:21.361857 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:21.362390 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:21.362394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:21.362394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:21.362585 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:21.362834 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:21.362835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:21.362835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:21.363440 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:21.363753 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:21.363834 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 06:37:21.364415 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":111,"threads_started":1}
I20260810 06:37:21.371587 10673 catalog_manager.cc:1347] Generated new cluster ID: c9f54258feb941c4a6456f48094a579f
I20260810 06:37:21.371651 10673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:37:21.379001 10673 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:37:21.380007 10673 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:37:21.385696 10673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e: Generated new TSK 0
I20260810 06:37:21.386019 10673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:37:22.302605 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:45915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35309
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:22.431752 10702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:22.432051 10702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:22.432090 10702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:22.433758 10702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:22.433825 10702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:22.433873 10702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:37:22.435791 10702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.71.20:45915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:37:22.436043 10702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:22.436329 10702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:22.443434 10707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:22.443688 10710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:22.443504 10708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:22.444730 10702 server_base.cc:1034] running on GCE node
I20260810 06:37:22.444999 10702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:22.445330 10702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:22.446532 10702 hybrid_clock.cc:648] HybridClock initialized: now 1786343842446501 us; error 45 us; skew 500 ppm
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:22.451062 10702 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:37:22.451725 10702 webserver.cc:460] Webserver started at http://127.30.71.1:45187/ using document root <none> and password file <none>
I20260810 06:37:22.451990 10702 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:22.452039 10702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:22.452188 10702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:37:22.468798 10702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "5af7c88e904e4ef0b5fecc912b646ea2"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "d40e8f0cfbfdc42dc80be807894b2a80"
server_key_iv: "aa05e69285778240089a503177b4db90"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:22.469367 10702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "5af7c88e904e4ef0b5fecc912b646ea2"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "d40e8f0cfbfdc42dc80be807894b2a80"
server_key_iv: "aa05e69285778240089a503177b4db90"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:22.472107 10702 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 06:37:22.495214 10720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:22.495569 10702 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260810 06:37:22.495738 10702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "5af7c88e904e4ef0b5fecc912b646ea2"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "d40e8f0cfbfdc42dc80be807894b2a80"
server_key_iv: "aa05e69285778240089a503177b4db90"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:22.495908 10702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:22.547236 10702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:22.549986 10702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:22.550247 10702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:22.550580 10702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:22.551203 10702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:22.551281 10702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:22.551333 10702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:22.551376 10702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:22.565557 10702 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38091
I20260810 06:37:22.565740 10846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38091 every 8 connection(s)
I20260810 06:37:22.566066 10702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:37:22.572844 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 10702
I20260810 06:37:22.572962 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:37:22.573164 31004 external_mini_cluster.cc:1428] Setting key fe24a526d1d7ee07e221c22da36100aa
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:37:22.579720 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:22.567171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35309 (local address 127.30.71.1:56899)
0810 06:37:22.567590 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:22.567635 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:22.568620 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:22.569028 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:22.569032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:22.569464 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:22.569906 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:22.569913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.571055 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.571057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:22.572526 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:22.572530 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.573135 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.573136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:22.573352 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:22.574074 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:22.574084 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:22.575853 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:22.577890 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:22.577897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:22.577900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:22.578222 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:22.578481 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:22.578482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:22.578482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:22.578622 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:22.578928 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:22.578934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:22.579166 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 06:37:22.579314 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 06:37:22.579478 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":200,"threads_started":1}
I20260810 06:37:22.580384 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:22.567161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:56899 (local address 127.30.71.62:35309)
0810 06:37:22.567282 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:22.567284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:22.567787 (+ 503us) server_negotiation.cc:407] Connection header received
0810 06:37:22.568780 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:22.568782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:22.568797 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:22.568888 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:22.570020 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.570878 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.572630 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.572979 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.573054 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:22.575974 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:22.575976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:22.575977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:22.575985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:22.577794 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:22.578334 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:22.578336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:22.578336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:22.578409 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:22.578696 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:22.578697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:22.578697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:22.578841 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:22.578902 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:22.580168 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0810 06:37:22.580276 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
I20260810 06:37:22.581204 10847 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35309
I20260810 06:37:22.581317 10847 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:22.581509 10847 heartbeater.cc:511] Master 127.30.71.62:35309 requested a full tablet report, sending...
I20260810 06:37:22.582155 10614 ts_manager.cc:194] Registered new tserver with Master: 5af7c88e904e4ef0b5fecc912b646ea2 (127.30.71.1:38091)
I20260810 06:37:22.583244 10614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:56899
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:37:22.607209 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:45915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35309
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 06:37:22.727912 10853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:22.728127 10853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:22.728219 10853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:22.729460 10853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:22.729525 10853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:22.729573 10853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:37:22.731276 10853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.71.20:45915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:37:22.731539 10853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:22.731792 10853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:22.739336 10860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:22.739881 10864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:22.740240 10853 server_base.cc:1034] running on GCE node
W20260810 06:37:22.740738 10861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:22.741030 10853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:22.741310 10853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:22.742483 10853 hybrid_clock.cc:648] HybridClock initialized: now 1786343842742462 us; error 33 us; skew 500 ppm
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:22.747210 10853 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:37:22.747874 10853 webserver.cc:460] Webserver started at http://127.30.71.2:36149/ using document root <none> and password file <none>
I20260810 06:37:22.748209 10853 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:22.748345 10853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:22.748548 10853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:37:22.762785 10853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "720151a60410417b837ceb06da0faa38"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "23f472b6eb198b381a4fcab5b7a3cc95"
server_key_iv: "e4361c55558eb33e85f66c728906099c"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:22.763578 10853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "720151a60410417b837ceb06da0faa38"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "23f472b6eb198b381a4fcab5b7a3cc95"
server_key_iv: "e4361c55558eb33e85f66c728906099c"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:22.768919 10853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:37:22.785601 10871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:22.785897 10853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:37:22.786005 10853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "720151a60410417b837ceb06da0faa38"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "23f472b6eb198b381a4fcab5b7a3cc95"
server_key_iv: "e4361c55558eb33e85f66c728906099c"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:22.786118 10853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:22.796130 10853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:22.798072 10853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:22.798235 10853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:22.798455 10853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:22.798866 10853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:22.798929 10853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:22.798995 10853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:22.799023 10853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:22.814239 10853 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:35367
I20260810 06:37:22.815054 10997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:35367 every 8 connection(s)
I20260810 06:37:22.815388 10853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:37:22.816135 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 10853
I20260810 06:37:22.816265 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:37:22.816484 31004 external_mini_cluster.cc:1428] Setting key 09de589cc133a1123065e09f9d89e6bf
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
I20260810 06:37:22.835824 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:22.820053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:51039 (local address 127.30.71.62:35309)
0810 06:37:22.820314 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:22.820320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:22.820376 (+ 56us) server_negotiation.cc:407] Connection header received
0810 06:37:22.821474 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:22.821476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:22.821492 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:22.821531 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:22.822500 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.823345 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.825092 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.825482 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.825593 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:22.829426 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:22.829434 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:22.829434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:22.829445 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:22.831642 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:22.832326 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:22.832327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:22.832328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:22.832394 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:22.832682 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:22.832683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:22.832683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:22.832878 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:22.832924 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:22.835553 (+ 2629us) server_negotiation.cc:299] Negotiation successful
0810 06:37:22.835677 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260810 06:37:22.836712 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:22.818296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35309 (local address 127.30.71.2:51039)
0810 06:37:22.820214 (+ 1918us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:22.820269 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:22.821300 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:22.821767 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:22.821771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:22.822056 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:22.822365 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:22.822371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.823476 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.823478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:22.824972 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:22.824976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:22.825601 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:22.825603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:22.825836 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:22.826809 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:22.826823 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:22.829187 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:22.831762 (+ 2575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:22.831770 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:22.831773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:22.832226 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:22.832467 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:22.832468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:22.832468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:22.832605 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:22.835117 (+ 2512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:22.835125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:22.835344 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 06:37:22.836226 (+ 882us) client_negotiation.cc:240] Negotiation successful
0810 06:37:22.836429 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1826,"thread_start_us":123,"threads_started":1}
I20260810 06:37:22.837430 10998 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35309
I20260810 06:37:22.837571 10998 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:22.837831 10998 heartbeater.cc:511] Master 127.30.71.62:35309 requested a full tablet report, sending...
I20260810 06:37:22.838238 10614 ts_manager.cc:194] Registered new tserver with Master: 720151a60410417b837ceb06da0faa38 (127.30.71.2:35367)
I20260810 06:37:22.838779 10614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:51039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:37:22.854346 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:45915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35309
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:37:22.974400 11005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:37:22.974633 11005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:37:22.974682 11005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:37:22.976008 11005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:37:22.976078 11005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:37:22.976125 11005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:37:22.977638 11005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.71.20:45915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:37:22.977909 11005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:37:22.978170 11005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:37:22.984012 11013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:22.984103 11010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:37:22.984371 11011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:37:22.984725 11005 server_base.cc:1034] running on GCE node
I20260810 06:37:22.984925 11005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:37:22.985189 11005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:37:22.986351 11005 hybrid_clock.cc:648] HybridClock initialized: now 1786343842986348 us; error 54 us; skew 500 ppm
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:37:22.989686 11005 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:37:22.990239 11005 webserver.cc:460] Webserver started at http://127.30.71.3:38029/ using document root <none> and password file <none>
I20260810 06:37:22.990514 11005 fs_manager.cc:359] Metadata directory not provided
I20260810 06:37:22.990598 11005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:37:22.990739 11005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:37:22 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:37:23.003517 11005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "77890c1bc98b4b33988069f045cf1a6d"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "382b32e0bef3005fe3ee283cdc5ea061"
server_key_iv: "9fcd4683fae5beb5ac8a80cb09957825"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:23.004051 11005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "77890c1bc98b4b33988069f045cf1a6d"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "382b32e0bef3005fe3ee283cdc5ea061"
server_key_iv: "9fcd4683fae5beb5ac8a80cb09957825"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:23.016402 11005 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.001s sys 0.001s
I20260810 06:37:23.035485 11020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:23.035792 11005 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.001s
I20260810 06:37:23.035900 11005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "77890c1bc98b4b33988069f045cf1a6d"
format_stamp: "Formatted at 2026-08-10 06:37:22 on dist-test-slave-w206"
server_key: "382b32e0bef3005fe3ee283cdc5ea061"
server_key_iv: "9fcd4683fae5beb5ac8a80cb09957825"
server_key_version: "kuduclusterkey@0"
I20260810 06:37:23.036010 11005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:37:23.048417 11005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:37:23.051149 11005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:37:23.051355 11005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:37:23.051662 11005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:37:23.052076 11005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:37:23.052148 11005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:23.052218 11005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:37:23.052253 11005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:37:23.069380 11005 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:39737
I20260810 06:37:23.069586 11141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:39737 every 8 connection(s)
I20260810 06:37:23.069986 11005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:37:23.072230 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 11005
I20260810 06:37:23.072355 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:37:23.072562 31004 external_mini_cluster.cc:1428] Setting key 120118ca94d92a75c9c40216f6748a4b
Aug 10 06:37:23 dist-test-slave-w206 krb5kdc[10095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:37:23.081568 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.071071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:35309 (local address 127.30.71.3:43573)
0810 06:37:23.071374 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.071421 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.071923 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.072356 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.072361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.072634 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:37:23.073029 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.073035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.073884 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.073885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.074777 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.074779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.075259 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.075261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.075429 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.076107 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:37:23.076138 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:37:23.078167 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:37:23.079975 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.079983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.079986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.080224 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.080457 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:37:23.080458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:37:23.080458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:37:23.080571 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:37:23.080849 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:37:23.080856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:37:23.081090 (+ 234us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.081211 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.081356 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":178,"threads_started":1}
I20260810 06:37:23.081561 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.071126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:43573 (local address 127.30.71.62:35309)
0810 06:37:23.071236 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.071238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.071528 (+ 290us) server_negotiation.cc:407] Connection header received
0810 06:37:23.072053 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.072054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.072079 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.072189 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:23.073194 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.073741 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.074885 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.075176 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.075253 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.078271 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:23.078273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:23.078273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:23.078281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:23.079860 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.080324 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.080325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.080326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.080388 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.080650 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.080651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.080652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.080773 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:23.080846 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.081287 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.081437 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:37:23.082161 11142 heartbeater.cc:347] Connected to a master server at 127.30.71.62:35309
I20260810 06:37:23.082310 11142 heartbeater.cc:464] Registering TS with master...
I20260810 06:37:23.082528 11142 heartbeater.cc:511] Master 127.30.71.62:35309 requested a full tablet report, sending...
I20260810 06:37:23.083101 10614 ts_manager.cc:194] Registered new tserver with Master: 77890c1bc98b4b33988069f045cf1a6d (127.30.71.3:39737)
I20260810 06:37:23.083639 10614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:43573
I20260810 06:37:23.084697 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:37:23.094153 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.086753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.30.71.62:35309)
0810 06:37:23.086799 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.086801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.087059 (+ 258us) server_negotiation.cc:407] Connection header received
0810 06:37:23.087132 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.087133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.087144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.087203 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:37:23.087819 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.088673 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.089475 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.089824 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.089901 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.091341 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:37:23.091342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:37:23.091343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:37:23.091350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:37:23.092829 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.093153 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.093154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.093154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.093258 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:37:23.093513 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:37:23.093514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:37:23.093514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:37:23.093695 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:37:23.093813 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.093934 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.094032 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:37:23.096114 10614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:37:23.097918 10614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:37:23.106952 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.101166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:35367 (local address 127.0.0.1:57292)
0810 06:37:23.102046 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.102055 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.102097 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.102659 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.102660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.102662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:23.102724 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.102726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.104163 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.104164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.105533 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.105536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.106634 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.106635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.106712 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.106726 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.106797 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.106854 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":150,"threads_started":1}
I20260810 06:37:23.107455 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.101341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57292 (local address 127.30.71.2:35367)
0810 06:37:23.102498 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.102501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.102509 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:37:23.102541 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.102543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.102556 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.102640 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:23.102787 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.104063 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.105653 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.106451 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.106522 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.106597 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.107295 (+ 698us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.107335 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":246,"threads_started":1}
I20260810 06:37:23.107574 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.101094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.30.71.3:39737)
0810 06:37:23.101355 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.101361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.101944 (+ 583us) server_negotiation.cc:407] Connection header received
0810 06:37:23.102042 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.102045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.102064 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.102160 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:23.103194 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.104394 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.106048 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.106791 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.106832 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.106911 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.107436 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.107484 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":158,"threads_started":1}
I20260810 06:37:23.107921 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.100875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39737 (local address 127.0.0.1:56364)
0810 06:37:23.101846 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.101871 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.101948 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.103053 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.103054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.103057 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:23.103094 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.103095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.104757 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.104758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.105838 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.105840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.107277 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.107277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.107339 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.107343 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.107740 (+ 397us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.107785 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":576,"threads_started":1}
I20260810 06:37:23.108489 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.100832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38091 (local address 127.0.0.1:34860)
0810 06:37:23.101515 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.101558 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.101672 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.102356 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.102358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.102360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:23.102416 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.102417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.104421 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.104429 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.106081 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.106085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.108014 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.108017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.108129 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.108134 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.108279 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.108337 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":170,"threads_started":1}
I20260810 06:37:23.108500 10925 tablet_service.cc:1467] Processing CreateTablet for tablet 7d32f59fc2f9472b8e5918937ccf66f8 (DEFAULT_TABLE table=test-table [id=366d0814761049ec87692c66a490020b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:23.108847 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.100934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.30.71.1:38091)
0810 06:37:23.101126 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.101132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.101628 (+ 496us) server_negotiation.cc:407] Connection header received
0810 06:37:23.101871 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.101874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.101903 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.102068 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:23.102496 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.104244 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.106204 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.107546 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.107664 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.107795 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.108626 (+ 831us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.108694 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260810 06:37:23.108902 10925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d32f59fc2f9472b8e5918937ccf66f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:23.111577 11070 tablet_service.cc:1467] Processing CreateTablet for tablet 7d32f59fc2f9472b8e5918937ccf66f8 (DEFAULT_TABLE table=test-table [id=366d0814761049ec87692c66a490020b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:23.111955 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d32f59fc2f9472b8e5918937ccf66f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:23.112687 11161 tablet_bootstrap.cc:492] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: Bootstrap starting.
I20260810 06:37:23.114952 11161 tablet_bootstrap.cc:654] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:23.115566 11161 log.cc:824] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:23.116093 10772 tablet_service.cc:1467] Processing CreateTablet for tablet 7d32f59fc2f9472b8e5918937ccf66f8 (DEFAULT_TABLE table=test-table [id=366d0814761049ec87692c66a490020b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:37:23.116489 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d32f59fc2f9472b8e5918937ccf66f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:37:23.116683 11161 tablet_bootstrap.cc:492] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: No bootstrap required, opened a new log
I20260810 06:37:23.116756 11161 ts_tablet_manager.cc:1397] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260810 06:37:23.119963 11161 raft_consensus.cc:348] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.120153 11161 raft_consensus.cc:374] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:23.120200 11161 raft_consensus.cc:729] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 720151a60410417b837ceb06da0faa38, State: Initialized, Role: FOLLOWER
I20260810 06:37:23.120349 11161 consensus_queue.cc:260] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.120720 11161 ts_tablet_manager.cc:1428] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260810 06:37:23.120872 10998 heartbeater.cc:503] Master 127.30.71.62:35309 was elected leader, sending a full tablet report...
I20260810 06:37:23.120957 11165 tablet_bootstrap.cc:492] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2: Bootstrap starting.
I20260810 06:37:23.121891 11162 tablet_bootstrap.cc:492] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Bootstrap starting.
I20260810 06:37:23.122105 11165 tablet_bootstrap.cc:654] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:23.122804 11165 log.cc:824] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:23.123505 11162 tablet_bootstrap.cc:654] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Neither blocks nor log segments found. Creating new log.
I20260810 06:37:23.123870 11165 tablet_bootstrap.cc:492] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2: No bootstrap required, opened a new log
I20260810 06:37:23.124006 11165 ts_tablet_manager.cc:1397] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:37:23.124388 11162 log.cc:824] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:37:23.126936 11165 raft_consensus.cc:348] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.127140 11165 raft_consensus.cc:374] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:23.127184 11165 raft_consensus.cc:729] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af7c88e904e4ef0b5fecc912b646ea2, State: Initialized, Role: FOLLOWER
I20260810 06:37:23.127350 11165 consensus_queue.cc:260] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.127737 11165 ts_tablet_manager.cc:1428] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 06:37:23.127758 11162 tablet_bootstrap.cc:492] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: No bootstrap required, opened a new log
I20260810 06:37:23.127818 11162 ts_tablet_manager.cc:1397] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 06:37:23.127861 10847 heartbeater.cc:503] Master 127.30.71.62:35309 was elected leader, sending a full tablet report...
I20260810 06:37:23.129832 11168 raft_consensus.cc:482] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:37:23.129926 11168 raft_consensus.cc:504] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.131493 11168 leader_election.cc:288] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77890c1bc98b4b33988069f045cf1a6d (127.30.71.3:39737), 720151a60410417b837ceb06da0faa38 (127.30.71.2:35367)
I20260810 06:37:23.131728 11162 raft_consensus.cc:348] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.131923 11162 raft_consensus.cc:374] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:37:23.131976 11162 raft_consensus.cc:729] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77890c1bc98b4b33988069f045cf1a6d, State: Initialized, Role: FOLLOWER
I20260810 06:37:23.132134 11162 consensus_queue.cc:260] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.132445 11162 ts_tablet_manager.cc:1428] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 06:37:23.132572 11142 heartbeater.cc:503] Master 127.30.71.62:35309 was elected leader, sending a full tablet report...
I20260810 06:37:23.135807 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.130892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:44121 (local address 127.30.71.3:39737)
0810 06:37:23.130939 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.130941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.131726 (+ 785us) server_negotiation.cc:407] Connection header received
0810 06:37:23.131844 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.131845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.131857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.131891 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:23.132180 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.132905 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.134628 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.135324 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.135381 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.135392 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.135697 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.135741 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:37:23.136075 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.130604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39737 (local address 127.30.71.1:44121)
0810 06:37:23.131655 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.131667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.131760 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.131973 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.131974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.131976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:23.132082 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.132085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.133061 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.133062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.134432 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.134435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.135461 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.135462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.135587 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.135591 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.135923 (+ 332us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.135962 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"thread_start_us":161,"threads_started":1}
I20260810 06:37:23.136476 11093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d32f59fc2f9472b8e5918937ccf66f8" candidate_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77890c1bc98b4b33988069f045cf1a6d" is_pre_election: true
I20260810 06:37:23.136716 11093 raft_consensus.cc:2454] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af7c88e904e4ef0b5fecc912b646ea2 in term 0.
I20260810 06:37:23.137001 10723 leader_election.cc:302] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af7c88e904e4ef0b5fecc912b646ea2, 77890c1bc98b4b33988069f045cf1a6d; no voters:
I20260810 06:37:23.137202 11169 raft_consensus.cc:2781] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:37:23.137295 11169 raft_consensus.cc:482] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:37:23.137331 11169 raft_consensus.cc:3037] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:23.138384 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.131374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:42061 (local address 127.30.71.2:35367)
0810 06:37:23.131424 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.131426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.133357 (+ 1931us) server_negotiation.cc:407] Connection header received
0810 06:37:23.133469 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.133470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.133485 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.133519 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:37:23.133920 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.134666 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.137128 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.137776 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.137829 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.137840 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.138252 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.138308 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:37:23.138798 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.131132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:35367 (local address 127.30.71.1:42061)
0810 06:37:23.133282 (+ 2150us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.133294 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.133386 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.133606 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:37:23.133607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:37:23.133609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:37:23.133820 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.133822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.134858 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.134859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.136088 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:37:23.136091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.137958 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.137959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:37:23.138119 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.138123 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:37:23.138528 (+ 405us) client_negotiation.cc:240] Negotiation successful
0810 06:37:23.138578 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1864,"thread_start_us":119,"threads_started":1}
I20260810 06:37:23.138748 11169 raft_consensus.cc:504] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.139083 11093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d32f59fc2f9472b8e5918937ccf66f8" candidate_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77890c1bc98b4b33988069f045cf1a6d"
I20260810 06:37:23.139169 11093 raft_consensus.cc:3037] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:23.139231 10943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d32f59fc2f9472b8e5918937ccf66f8" candidate_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "720151a60410417b837ceb06da0faa38" is_pre_election: true
I20260810 06:37:23.139467 10943 raft_consensus.cc:2454] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af7c88e904e4ef0b5fecc912b646ea2 in term 0.
I20260810 06:37:23.140162 11093 raft_consensus.cc:2454] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af7c88e904e4ef0b5fecc912b646ea2 in term 1.
I20260810 06:37:23.140363 10723 leader_election.cc:302] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af7c88e904e4ef0b5fecc912b646ea2, 77890c1bc98b4b33988069f045cf1a6d; no voters:
I20260810 06:37:23.140461 11168 raft_consensus.cc:2781] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:37:23.140617 11168 raft_consensus.cc:686] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 5af7c88e904e4ef0b5fecc912b646ea2, State: Running, Role: LEADER
I20260810 06:37:23.140722 11168 consensus_queue.cc:237] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } }
I20260810 06:37:23.141578 11169 leader_election.cc:288] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [CANDIDATE]: Term 1 election: Requested vote from peers 77890c1bc98b4b33988069f045cf1a6d (127.30.71.3:39737), 720151a60410417b837ceb06da0faa38 (127.30.71.2:35367)
I20260810 06:37:23.141621 10614 catalog_manager.cc:5495] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af7c88e904e4ef0b5fecc912b646ea2 (127.30.71.1). New cstate: current_term: 1 leader_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 } health_report { overall_health: UNKNOWN } } }
I20260810 06:37:23.141778 10943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d32f59fc2f9472b8e5918937ccf66f8" candidate_uuid: "5af7c88e904e4ef0b5fecc912b646ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "720151a60410417b837ceb06da0faa38"
I20260810 06:37:23.141871 10943 raft_consensus.cc:3037] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:37:23.144232 10943 raft_consensus.cc:2454] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af7c88e904e4ef0b5fecc912b646ea2 in term 1.
I20260810 06:37:23.147935 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.145386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.30.71.1:38091)
0810 06:37:23.145426 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:37:23.145428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:37:23.145453 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:37:23.145540 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:37:23.145541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:37:23.145559 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:37:23.145647 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:37:23.145760 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.146219 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.147029 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:37:23.147226 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:37:23.147313 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:37:23.147497 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:37:23.147596 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:37:23.147691 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:37:23.147709 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 06:37:23.147731 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:37:23.150238 11093 raft_consensus.cc:1261] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 1 FOLLOWER]: Refusing update from remote peer 5af7c88e904e4ef0b5fecc912b646ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:23.150239 10943 raft_consensus.cc:1261] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 1 FOLLOWER]: Refusing update from remote peer 5af7c88e904e4ef0b5fecc912b646ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:37:23.150591 11169 consensus_queue.cc:1035] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77890c1bc98b4b33988069f045cf1a6d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:23.150765 11173 consensus_queue.cc:1035] T 7d32f59fc2f9472b8e5918937ccf66f8 P 5af7c88e904e4ef0b5fecc912b646ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "720151a60410417b837ceb06da0faa38" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 35367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:37:23.153976 11176 mvcc.cc:205] Tried to move back new op lower bound from 7316864381539561472 to 7316864381505089536. Current Snapshot: MvccSnapshot[applied={T|T < 7316864381539561472}]
I20260810 06:37:23.156236 11178 mvcc.cc:205] Tried to move back new op lower bound from 7316864381539561472 to 7316864381505089536. Current Snapshot: MvccSnapshot[applied={T|T < 7316864381539561472}]
I20260810 06:37:23.156613 11177 mvcc.cc:205] Tried to move back new op lower bound from 7316864381539561472 to 7316864381505089536. Current Snapshot: MvccSnapshot[applied={T|T < 7316864381539561472}]
I20260810 06:37:23.159197 10613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:23.159312 10613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:37:23.160162 10613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c183dcc63384583857f1b36dfa13c2e: Sending DeleteTablet for 3 replicas of tablet 7d32f59fc2f9472b8e5918937ccf66f8
I20260810 06:37:23.160789 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d32f59fc2f9472b8e5918937ccf66f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:23 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:34860
I20260810 06:37:23.160993 11070 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d32f59fc2f9472b8e5918937ccf66f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:23 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:56364
I20260810 06:37:23.161137 10925 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d32f59fc2f9472b8e5918937ccf66f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:37:23 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:57292
I20260810 06:37:23.161556 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 10702
I20260810 06:37:23.163887 11188 tablet_replica.cc:330] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: stopping tablet replica
I20260810 06:37:23.164079 11188 raft_consensus.cc:2229] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:23.164318 11188 raft_consensus.cc:2258] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:23.164902 11188 ts_tablet_manager.cc:1905] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:23.167163 11187 tablet_replica.cc:330] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: stopping tablet replica
I20260810 06:37:23.167346 11187 raft_consensus.cc:2229] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:37:23.167645 11187 raft_consensus.cc:2258] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:37:23.168174 11187 ts_tablet_manager.cc:1905] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:37:23.170432 11187 ts_tablet_manager.cc:1918] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:23.170519 11187 log.cc:1196] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/7d32f59fc2f9472b8e5918937ccf66f8
I20260810 06:37:23.171181 11187 ts_tablet_manager.cc:1939] T 7d32f59fc2f9472b8e5918937ccf66f8 P 77890c1bc98b4b33988069f045cf1a6d: Deleting consensus metadata
I20260810 06:37:23.171682 10597 catalog_manager.cc:4841] TS 77890c1bc98b4b33988069f045cf1a6d (127.30.71.3:39737): tablet 7d32f59fc2f9472b8e5918937ccf66f8 (table test-table [id=366d0814761049ec87692c66a490020b]) successfully deleted
W20260810 06:37:23.172088 10598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:38091 (error 108)
I20260810 06:37:23.172309 11188 ts_tablet_manager.cc:1918] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:37:23.172405 11188 log.cc:1196] T 7d32f59fc2f9472b8e5918937ccf66f8 P 720151a60410417b837ceb06da0faa38: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/7d32f59fc2f9472b8e5918937ccf66f8
I20260810 06:37:23.172945 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 10853
I20260810 06:37:23.185245 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:37:23.185110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38091 (local address 127.0.0.1:34884)
0810 06:37:23.185165 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.185197 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:37:23.185423 10598 catalog_manager.cc:4584] TS 5af7c88e904e4ef0b5fecc912b646ea2 (127.30.71.1:38091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d32f59fc2f9472b8e5918937ccf66f8: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38091: connect: Connection refused (error 111)
W20260810 06:37:23.186087 11159 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:37:23.185740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:35367 (local address 127.0.0.1:57302)
0810 06:37:23.185780 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 06:37:23.185790 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:37:23.185831 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:37:23.186043 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.2:35367: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 06:37:23.186676 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 11005
I20260810 06:37:23.195793 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 10575
I20260810 06:37:23.203449 31004 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:37:23.547899 31004 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:37:23.548022 31004 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:37:23.893606 31004 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:37:23.893 UTC [10127] LOG: received smart shutdown request
2026-08-10 06:37:23.897 UTC [10127] LOG: background worker "logical replication launcher" (PID 10136) exited with exit code 1
2026-08-10 06:37:23.898 UTC [10131] LOG: shutting down
2026-08-10 06:37:24.081 UTC [10127] LOG: database system is shut down
2026-08-10T06:37:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50955 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:37:24 dist-test-slave-w206 krb5kdc[11202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:37:24 dist-test-slave-w206 krb5kdc[11202](info): set up 2 sockets
Aug 10 06:37:24 dist-test-slave-w206 krb5kdc[11202](info): commencing operation
krb5kdc: starting...
W20260810 06:37:25.301103 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:37:25 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:37:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/rangeradmin_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/rangeradmin_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/rangerlookup_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/rangerlookup_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
I20260810 06:37:25.486135 31004 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskI3NzPm/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/postgres -l logfile start
2026-08-10 06:37:28.066 UTC [11232] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:37:28.066 UTC [11232] LOG: listening on IPv4 address "127.30.71.20", port 56043
2026-08-10 06:37:28.070 UTC [11232] LOG: listening on Unix socket "/tmp/.s.PGSQL.56043"
2026-08-10 06:37:28.092 UTC [11237] LOG: database system was shut down at 2026-08-10 06:37:26 UTC
2026-08-10 06:37:28.095 UTC [11232] LOG: database system is ready to accept connections
I20260810 06:37:29.159408 31004 mini_postgres.cc:96] Postgres bound to 56043
2026-08-10 06:37:29.167 UTC [11245] FATAL: database "slave" does not exist
127.30.71.20:56043 - accepting connections
I20260810 06:37:29.167799 31004 mini_ranger.cc:161] Starting Ranger...
I20260810 06:37:29.184000 31004 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:37:29.667785 31004 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:37:29.667896 31004 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-admin
2026-08-10 06:37:29,951 [I] DB FLAVOR :POSTGRES
2026-08-10 06:37:29,951 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:37:29,951 [I] Checking connection..
2026-08-10 06:37:29,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:37:30,215 [I] Checking connection passed.
2026-08-10 06:37:30,215 [I] --------- Verifying version history table ---------
2026-08-10 06:37:30,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:37:30,462 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:37:30,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:37:30,703 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:37:30,704 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:37:30,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:37:30.930 UTC [11315] WARNING: there is no transaction in progress
2026-08-10 06:37:30,937 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:37:30,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:37:31,198 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:37:31,198 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:37:31,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:37:31,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:37:31,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w206.c.gcp-upstream.internal','N') ;"
2026-08-10 06:37:31,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:37:32,178 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:37:32,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:37:32,445 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:37:32,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:37:32,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:37:32,953 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:37:32,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:37:33.912 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:33.928 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:33.943 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:33.958 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:33.973 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:33.991 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.047 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.054 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.068 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.079 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.089 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.102 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.112 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.119 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.126 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34.133 UTC [11434] WARNING: there is no transaction in progress
2026-08-10 06:37:34,781 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:37:34,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:37:35,036 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:37:35,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:37:35,302 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:37:35,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:37:35,564 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:37:35,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:37:35,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:37:36,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w206.c.gcp-upstream.internal';"
2026-08-10 06:37:36,335 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:37:36,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:37:36,596 [I] DB_PATCHES have already been applied
I20260810 06:37:36.601691 31004 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-admin:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskI3NzPm/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:37:36.601756 31004 mini_ranger.cc:193] Using host: 127.30.71.20
I20260810 06:37:36.605252 31004 mini_ranger.cc:227] Ranger admin URL: http://127.30.71.20:43703
Aug 10, 2026 6:37:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:37:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:37:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:37:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:37:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:37:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:37:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:37:37.776250 31004 mini_ranger.cc:160] Time spent starting Ranger: real 8.608s user 0.001s sys 0.008s
Aug 10, 2026 6:37:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:37:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:37:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:37:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:37:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:37:59.558444 31004 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.71.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.71.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/rangerkms_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.71.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/rangerkms_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.71.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.71.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.71.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/HTTP_127.30.71.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:37:59 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343879, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:37:59.651300 31004 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:37:59.662118 31004 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:38:00.623734 31004 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:38:00.623836 31004 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms
I20260810 06:38:00.623862 31004 mini_ranger_kms.cc:227] Using postgres at 127.30.71.20:56043
2026-08-10 06:38:00,806 [I] DB FLAVOR :POSTGRES
2026-08-10 06:38:00,807 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:38:00,807 [I] Checking connection
2026-08-10 06:38:00,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:38:01,077 [I] connection success
2026-08-10 06:38:01,077 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:38:01,077 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:38:01,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:38:01,316 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:38:01,317 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:38:01,317 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:38:01,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.71.20:56043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:38:01,594 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:38:02.101658 31004 mini_ranger_kms.cc:310] Created kms service
I20260810 06:38:02.282747 31004 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:38:02.329725 31004 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:38:02 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343879, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:02.730748 31004 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:38:02.731022 31004 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms:/tmp/dist-test-taskI3NzPm/build/release/bin/postgresql.jar:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskI3NzPm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:38:02.731058 31004 mini_ranger_kms.cc:240] Using host: 127.30.71.20
I20260810 06:38:02.735339 31004 mini_ranger_kms.cc:276] Ranger KMS PID: 11627
I20260810 06:38:02.735430 31004 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.71.20:49653
Aug 10, 2026 6:38:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:38:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:38:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:38:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:38:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:38:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:38:05 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.71.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:05.173928 31004 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.523s user 0.000s sys 0.167s
I20260810 06:38:05.174045 31004 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:38:05.174091 31004 mini_ranger_kms.cc:401] 127.30.71.20:49653/kms/v1/keys
Aug 10 06:38:05 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.71.20@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
Aug 10 06:38:05 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343885, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.71.20@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
Aug 10 06:38:08 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:38:08.330156 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:45631
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:45631
--ranger_config_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:08.417390 11682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:08.417618 11682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:08.417642 11682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:08.418766 11682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:08.418802 11682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:08.418817 11682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:08.418829 11682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:08.418843 11682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:08.420400 11682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.71.20:49653/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:45631
--ranger_config_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:45631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:08.420596 11682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:08.420768 11682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:08.426257 11691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:08.426263 11688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:08.426298 11682 server_base.cc:1034] running on GCE node
W20260810 06:38:08.426309 11687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:08.426717 11682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:08.427101 11682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:08.428259 11682 hybrid_clock.cc:648] HybridClock initialized: now 1786343888428240 us; error 41 us; skew 500 ppm
Aug 10 06:38:08 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:08.431517 11682 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:08.432209 11682 webserver.cc:460] Webserver started at http://127.30.71.62:46787/ using document root <none> and password file <none>
I20260810 06:38:08.432543 11682 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:08.432694 11682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:08.432928 11682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:38:08 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:08.507031 11682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "8f5ee9d93db84b93b3130cd6929463cb"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-w206"
server_key: "7cf4e5f2baaf480e828cada48ff44519"
server_key_iv: "18fc067c2ca96ed37bfc53780801ea4e"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:08.507544 11682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "8f5ee9d93db84b93b3130cd6929463cb"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-w206"
server_key: "7cf4e5f2baaf480e828cada48ff44519"
server_key_iv: "18fc067c2ca96ed37bfc53780801ea4e"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:08.509104 11682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:38:08.531900 11699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:08.532217 11682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:08.532388 11682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "8f5ee9d93db84b93b3130cd6929463cb"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-w206"
server_key: "7cf4e5f2baaf480e828cada48ff44519"
server_key_iv: "18fc067c2ca96ed37bfc53780801ea4e"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:08.532590 11682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:08.548996 11682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:08.551107 11682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:08.551275 11682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:08.556488 11682 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:45631
I20260810 06:38:08.556517 11752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:45631 every 8 connection(s)
I20260810 06:38:08.556993 11682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:08.557736 11753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:08.558724 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 11682
I20260810 06:38:08.558831 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:38:08.559057 31004 external_mini_cluster.cc:1428] Setting key 56decfd890856224a8a6878ea5de6f33
I20260810 06:38:08.560596 11753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Bootstrap starting.
I20260810 06:38:08.561424 11753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:08.561774 11753 log.cc:824] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:08.562618 11753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: No bootstrap required, opened a new log
I20260810 06:38:08.565184 11753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:08.565385 11753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:08.565465 11753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5ee9d93db84b93b3130cd6929463cb, State: Initialized, Role: FOLLOWER
I20260810 06:38:08.565626 11753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:08.565729 11753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:08.565786 11753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:08.565868 11753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:08.566499 11753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:08.566644 11753 leader_election.cc:302] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f5ee9d93db84b93b3130cd6929463cb; no voters:
I20260810 06:38:08.566833 11753 leader_election.cc:288] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:08.566886 11759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:08.567183 11759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 LEADER]: Becoming Leader. State: Replica: 8f5ee9d93db84b93b3130cd6929463cb, State: Running, Role: LEADER
Aug 10 06:38:08 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:08.567274 11753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:08.567821 11759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:08.568277 11759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f5ee9d93db84b93b3130cd6929463cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } } }
I20260810 06:38:08.568416 11759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:08.568588 11759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5ee9d93db84b93b3130cd6929463cb. Latest consensus state: current_term: 1 leader_uuid: "8f5ee9d93db84b93b3130cd6929463cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } } }
I20260810 06:38:08.568683 11759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:08.568804 11753 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:38:08.568830 11776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:08.569433 11776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:08.576150 11757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:08.559628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.30.71.62:45631)
0810 06:38:08.559816 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:08.559826 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:08.559857 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:38:08.560546 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:08.560555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:08.560620 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:08.560769 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:08.561646 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.562902 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.564007 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:08.564623 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:08.564756 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:08.567832 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:08.567838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:08.567838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:08.567852 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:08.570650 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.571104 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.571108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.571109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.571208 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:08.571440 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:08.571440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:08.571441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:08.571779 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:08.574505 (+ 2726us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:08.574739 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 06:38:08.575670 (+ 931us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260810 06:38:08.576632 11776 catalog_manager.cc:1347] Generated new cluster ID: ef9e70219dbb447c8d1e52ed6a2f2973
I20260810 06:38:08.577011 11776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:08.607322 11776 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:08.608135 11776 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:08.616829 11776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Generated new TSK 0
I20260810 06:38:08.617056 11776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:38:09.315421 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45631
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:09.410701 11804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:09.410935 11804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:09.410960 11804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:09.412319 11804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:09.412369 11804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:09.412395 11804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:38:09.413902 11804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.71.20:49653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:09.414160 11804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:09.414448 11804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:09.419727 11811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.419749 11810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.420174 11813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:09.420754 11804 server_base.cc:1034] running on GCE node
I20260810 06:38:09.420993 11804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:09.421317 11804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:09.422479 11804 hybrid_clock.cc:648] HybridClock initialized: now 1786343889422460 us; error 36 us; skew 500 ppm
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:09.425190 11804 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:38:09.425731 11804 webserver.cc:460] Webserver started at http://127.30.71.1:40419/ using document root <none> and password file <none>
I20260810 06:38:09.425984 11804 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:09.426030 11804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:09.426162 11804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:09.441053 11804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "d6a5fe508d214b5294f4ed685585cdae"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "3d010c36bc70253ba2162e0e96f87e8f"
server_key_iv: "131f32c770ac2647f68daba8f8693885"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.441553 11804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "d6a5fe508d214b5294f4ed685585cdae"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "3d010c36bc70253ba2162e0e96f87e8f"
server_key_iv: "131f32c770ac2647f68daba8f8693885"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.444020 11804 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:38:09.460247 11820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.460525 11804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:09.460640 11804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "d6a5fe508d214b5294f4ed685585cdae"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "3d010c36bc70253ba2162e0e96f87e8f"
server_key_iv: "131f32c770ac2647f68daba8f8693885"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.460742 11804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:09.478344 11804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:09.480868 11804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:09.481024 11804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:09.481320 11804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:09.481696 11804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:09.481745 11804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.481775 11804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:09.481793 11804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.489473 11804 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:44707
I20260810 06:38:09.490026 11804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:09.490460 11942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:44707 every 8 connection(s)
I20260810 06:38:09.493009 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 11804
I20260810 06:38:09.493103 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:09.493285 31004 external_mini_cluster.cc:1428] Setting key 172b261c965a0f11883c0424bcd254a5
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:09.513955 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.491787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:47003 (local address 127.30.71.62:45631)
0810 06:38:09.491998 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.492000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.496136 (+ 4136us) server_negotiation.cc:407] Connection header received
0810 06:38:09.496168 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.496169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.496183 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.496274 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.497050 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.497839 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.505473 (+ 7634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.505792 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.505872 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.510591 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.510593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.510594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.510602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.512290 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.512824 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.512825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.512826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.512899 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.513183 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.513184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.513185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.513315 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.513394 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.513736 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.513837 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":115,"threads_started":1}
I20260810 06:38:09.514115 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.491430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45631 (local address 127.30.71.1:47003)
0810 06:38:09.495259 (+ 3829us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.495316 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.495986 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.496402 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.496407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.496663 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.496955 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.496962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.504053 (+ 7091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.504056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.505320 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.505325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.506203 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.506204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.506384 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.507098 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.507110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.510460 (+ 3350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.512409 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.512418 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.512421 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.512717 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.512969 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.512971 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.512971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.513107 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.513397 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.513406 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.513588 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.513699 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.513863 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3760,"thread_start_us":89,"threads_started":1}
I20260810 06:38:09.514659 11943 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45631
I20260810 06:38:09.514791 11943 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.515036 11943 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:09.515600 11716 ts_manager.cc:194] Registered new tserver with Master: d6a5fe508d214b5294f4ed685585cdae (127.30.71.1:44707)
I20260810 06:38:09.516315 11716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:47003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:38:09.527035 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45631
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:09.649065 11951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:09.649428 11951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:09.649478 11951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:09.651613 11951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:09.651695 11951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:09.651744 11951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:38:09.654438 11951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.71.20:49653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:09.654733 11951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:09.655167 11951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:09.661988 11960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.662118 11961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.662253 11963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:09.662784 11951 server_base.cc:1034] running on GCE node
I20260810 06:38:09.663074 11951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:09.663429 11951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:09.664654 11951 hybrid_clock.cc:648] HybridClock initialized: now 1786343889664627 us; error 38 us; skew 500 ppm
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:09.668417 11951 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:38:09.669095 11951 webserver.cc:460] Webserver started at http://127.30.71.2:39509/ using document root <none> and password file <none>
I20260810 06:38:09.669489 11951 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:09.669627 11951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:09.669831 11951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:09.689567 11951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "78653496534848b28d4a59d4f8322656"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "861c4af66af160ab7c8b854a5bb00e90"
server_key_iv: "625bad2264e7b49333825c052b966462"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.690210 11951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "78653496534848b28d4a59d4f8322656"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "861c4af66af160ab7c8b854a5bb00e90"
server_key_iv: "625bad2264e7b49333825c052b966462"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.695932 11951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:09.720958 11970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.721246 11951 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:09.721342 11951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "78653496534848b28d4a59d4f8322656"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "861c4af66af160ab7c8b854a5bb00e90"
server_key_iv: "625bad2264e7b49333825c052b966462"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.721438 11951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:09.736292 11951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:09.739519 11951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:09.739771 11951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:09.740105 11951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:09.740598 11951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:09.740713 11951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.740803 11951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:09.740876 11951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:09.748435 11951 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:41769
I20260810 06:38:09.749009 11951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:09.758771 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 11951
I20260810 06:38:09.758883 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:09.759131 31004 external_mini_cluster.cc:1428] Setting key ac3660dc40db4a8156a1af60719a24ba
I20260810 06:38:09.763439 12091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:41769 every 8 connection(s)
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:09.782845 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.764690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:52673 (local address 127.30.71.62:45631)
0810 06:38:09.764747 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:09.764749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:09.771337 (+ 6588us) server_negotiation.cc:407] Connection header received
0810 06:38:09.772236 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:09.772238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:09.772251 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:09.772283 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:09.773208 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.774085 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.775869 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.776200 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.776245 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.779405 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:09.779407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:09.779407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:09.779417 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:09.781205 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.781744 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.781745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.781746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.781804 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:09.782065 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:09.782066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:09.782066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:09.782202 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:09.782255 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:09.782623 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 06:38:09.782736 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:38:09.783440 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:09.764324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45631 (local address 127.30.71.2:52673)
0810 06:38:09.771137 (+ 6813us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:09.771207 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:09.772109 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:09.772474 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:09.772478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:09.772747 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:09.773036 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.773042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.774214 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.774215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.775727 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:09.775731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:09.776378 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:09.776379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:09.776535 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:09.777220 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:09.777231 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:09.779269 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:09.781316 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.781326 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.781329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.781646 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.781869 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:09.781870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:09.781870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:09.781999 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:09.782284 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:09.782289 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:09.782478 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 06:38:09.782957 (+ 479us) client_negotiation.cc:240] Negotiation successful
0810 06:38:09.783194 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6746,"thread_start_us":127,"threads_started":1}
I20260810 06:38:09.784219 12092 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45631
I20260810 06:38:09.784356 12092 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:09.784598 12092 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:09.785019 11716 ts_manager.cc:194] Registered new tserver with Master: 78653496534848b28d4a59d4f8322656 (127.30.71.2:41769)
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
I20260810 06:38:09.785486 11716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:52673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:38:09.808485 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45631
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:09.949136 12098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:09.949559 12098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:09.949668 12098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:09.951603 12098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:09.951851 12098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:09.952126 12098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:38:09.954566 12098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.71.20:49653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:09.955108 12098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:09.955493 12098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:09.963258 12106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:09.963558 12098 server_base.cc:1034] running on GCE node
W20260810 06:38:09.963251 12104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:09.966004 12108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:09.966413 12098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:09.966708 12098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:09.967877 12098 hybrid_clock.cc:648] HybridClock initialized: now 1786343889967849 us; error 43 us; skew 500 ppm
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:09.971658 12098 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:38:09.972209 12098 webserver.cc:460] Webserver started at http://127.30.71.3:41763/ using document root <none> and password file <none>
I20260810 06:38:09.972472 12098 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:09.972525 12098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:09.972654 12098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:38:09 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:09.992775 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "c9cb3a92ec614ff6882c8e532ef7525d"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "90a993b1dbc7ed36439e0e799a678974"
server_key_iv: "71ae01ded22dcc328e4fcecd941a37af"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.993259 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "c9cb3a92ec614ff6882c8e532ef7525d"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "90a993b1dbc7ed36439e0e799a678974"
server_key_iv: "71ae01ded22dcc328e4fcecd941a37af"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:09.998275 12098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260810 06:38:10.026293 12115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.026577 12098 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260810 06:38:10.026696 12098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "c9cb3a92ec614ff6882c8e532ef7525d"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "90a993b1dbc7ed36439e0e799a678974"
server_key_iv: "71ae01ded22dcc328e4fcecd941a37af"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:10.026808 12098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:10.044829 12098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:10.047935 12098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:10.048122 12098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:10.048453 12098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:10.048897 12098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:10.048954 12098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.048990 12098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:10.049019 12098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.067800 12098 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:35929
I20260810 06:38:10.068280 12098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:10.068894 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 12098
I20260810 06:38:10.068998 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:10.069219 31004 external_mini_cluster.cc:1428] Setting key ba83b99bf1edc71c69b42453b04da35e
I20260810 06:38:10.079308 12238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:35929 every 8 connection(s)
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:10.094067 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.074024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:39337 (local address 127.30.71.62:45631)
0810 06:38:10.074067 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.074069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.075286 (+ 1217us) server_negotiation.cc:407] Connection header received
0810 06:38:10.076069 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.076071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.076084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.076114 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.077003 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.077799 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.079375 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.079670 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.079726 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.082589 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.082590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.082591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.082598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.087032 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.092641 (+ 5609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.092643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.092643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.092761 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.093141 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.093142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.093142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.093305 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.093368 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.093834 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.093954 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:38:10.095273 12241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.073211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45631 (local address 127.30.71.3:39337)
0810 06:38:10.075147 (+ 1936us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.075198 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.075942 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.076308 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.076313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.076583 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:10.076883 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.076889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.077942 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.077943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.079270 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.079274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.079838 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.079839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.080031 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.080666 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:10.080682 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:10.082473 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:10.087296 (+ 4823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.087308 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.087310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.087654 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.092854 (+ 5200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.092854 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.092855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.093055 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.093412 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:10.093421 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:10.093640 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.094779 (+ 1139us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.095018 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1893,"thread_start_us":594,"threads_started":1}
I20260810 06:38:10.095988 12239 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45631
I20260810 06:38:10.096083 12239 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.096284 12239 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:10.096724 11716 ts_manager.cc:194] Registered new tserver with Master: c9cb3a92ec614ff6882c8e532ef7525d (127.30.71.3:35929)
I20260810 06:38:10.097222 11716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:39337
I20260810 06:38:10.104050 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:10.104763 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 11804
I20260810 06:38:10.132189 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 11951
I20260810 06:38:10.147607 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 12098
I20260810 06:38:10.156294 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 11682
I20260810 06:38:10.166807 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:45631
--webserver_interface=127.30.71.62
--webserver_port=46787
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:45631
--ranger_config_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:10.254844 12242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:10.255125 12242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:10.255153 12242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:10.256325 12242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:10.256366 12242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:10.256381 12242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:10.256393 12242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:10.256405 12242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:10.258013 12242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:45631
--ranger_config_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:45631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=46787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:10.258272 12242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:10.258500 12242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:10.263494 12250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.263494 12248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:10.263528 12242 server_base.cc:1034] running on GCE node
W20260810 06:38:10.263520 12247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:10.264016 12242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:10.264266 12242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:10.265442 12242 hybrid_clock.cc:648] HybridClock initialized: now 1786343890265429 us; error 30 us; skew 500 ppm
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:10.268167 12242 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:10.268661 12242 webserver.cc:460] Webserver started at http://127.30.71.62:46787/ using document root <none> and password file <none>
I20260810 06:38:10.268906 12242 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:10.268962 12242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:10.349445 12242 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:10.350363 12258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.350615 12242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:10.350725 12242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "8f5ee9d93db84b93b3130cd6929463cb"
format_stamp: "Formatted at 2026-08-10 06:38:08 on dist-test-slave-w206"
server_key: "7cf4e5f2baaf480e828cada48ff44519"
server_key_iv: "18fc067c2ca96ed37bfc53780801ea4e"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:10.351292 12242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:10.372800 12242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:10.374619 12242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:10.374744 12242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:10.379263 12242 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:45631
I20260810 06:38:10.379302 12313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:45631 every 8 connection(s)
I20260810 06:38:10.379714 12242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:10.381026 12314 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:38:10.381688 12314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Bootstrap starting.
I20260810 06:38:10.384953 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 12242
I20260810 06:38:10.385269 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:44707
--local_ip_for_outbound_sockets=127.30.71.1
--tserver_master_addrs=127.30.71.62:45631
--webserver_port=40419
--webserver_interface=127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:10.389915 12314 log.cc:824] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:10.391826 12314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:10.392130 12314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Bootstrap complete.
I20260810 06:38:10.394613 12314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:10.394884 12314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5ee9d93db84b93b3130cd6929463cb, State: Initialized, Role: FOLLOWER
I20260810 06:38:10.395123 12314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:10.395222 12314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:10.395277 12314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:10.395325 12314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:38:10.396382 12314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:10.396503 12314 leader_election.cc:302] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f5ee9d93db84b93b3130cd6929463cb; no voters:
I20260810 06:38:10.396715 12314 leader_election.cc:288] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:38:10.396754 12318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:38:10.396888 12318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [term 2 LEADER]: Becoming Leader. State: Replica: 8f5ee9d93db84b93b3130cd6929463cb, State: Running, Role: LEADER
I20260810 06:38:10.397006 12314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:10.397058 12318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } }
I20260810 06:38:10.397423 12320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5ee9d93db84b93b3130cd6929463cb. Latest consensus state: current_term: 2 leader_uuid: "8f5ee9d93db84b93b3130cd6929463cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } } }
I20260810 06:38:10.397527 12320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:10.397437 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8f5ee9d93db84b93b3130cd6929463cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ee9d93db84b93b3130cd6929463cb" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 45631 } } }
I20260810 06:38:10.397601 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:10.397869 12323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:10.398476 12323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:10.398649 12323 catalog_manager.cc:1259] Loaded cluster ID: ef9e70219dbb447c8d1e52ed6a2f2973
I20260810 06:38:10.398708 12323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:10.398855 12314 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:38:10.402336 12323 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:10.403096 12323 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Loaded TSK: 0
I20260810 06:38:10.403442 12323 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:38:10.509447 12316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:10.509814 12316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:10.509887 12316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:10.511811 12316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:10.511948 12316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:10.512048 12316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:38:10.514372 12316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:44707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=40419
--tserver_master_addrs=127.30.71.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:10.514914 12316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:10.515302 12316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:10.521840 12357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.521975 12360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.521848 12358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:10.523316 12316 server_base.cc:1034] running on GCE node
I20260810 06:38:10.523581 12316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:10.523901 12316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:10.525096 12316 hybrid_clock.cc:648] HybridClock initialized: now 1786343890525071 us; error 40 us; skew 500 ppm
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:10.528699 12316 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:38:10.529346 12316 webserver.cc:460] Webserver started at http://127.30.71.1:40419/ using document root <none> and password file <none>
I20260810 06:38:10.529655 12316 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:10.529753 12316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:10.553850 12316 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:10.554802 12370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.555084 12316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:10.555235 12316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "d6a5fe508d214b5294f4ed685585cdae"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "3d010c36bc70253ba2162e0e96f87e8f"
server_key_iv: "131f32c770ac2647f68daba8f8693885"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:10.555685 12316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:10.574146 12316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:10.576566 12316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:10.576759 12316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:10.577056 12316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:10.577399 12316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:10.577457 12316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.577502 12316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:10.577545 12316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.584309 12316 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:44707
I20260810 06:38:10.584414 12491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:44707 every 8 connection(s)
I20260810 06:38:10.584791 12316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:10.586284 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 12316
I20260810 06:38:10.586591 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:41769
--local_ip_for_outbound_sockets=127.30.71.2
--tserver_master_addrs=127.30.71.62:45631
--webserver_port=39509
--webserver_interface=127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:10.598309 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.585783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45631 (local address 127.30.71.1:60041)
0810 06:38:10.585994 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.586049 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.586813 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.587568 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.587574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.587851 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:10.588154 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.588162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.589562 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.589563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.590884 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.590887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.591579 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.591580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.591726 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.592324 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:10.592333 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:10.593679 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:10.595733 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.595740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.595742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.595993 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.596284 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.596284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.596285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.596423 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.597009 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:10.597015 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:10.597162 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.597290 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.597464 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260810 06:38:10.598467 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.585887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:60041 (local address 127.30.71.62:45631)
0810 06:38:10.586139 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.586149 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.586203 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:38:10.587092 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.587100 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.587164 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.587504 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.588313 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.589453 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.591020 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.591463 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.591594 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.593834 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.593839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.593839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.593849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.595618 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.596096 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.596099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.596100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.596204 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.596548 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.596550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.596550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.596874 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.596963 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.597705 (+ 742us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.597853 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":113,"threads_started":1}
I20260810 06:38:10.599260 12492 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45631
I20260810 06:38:10.599375 12492 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.599591 12492 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
W20260810 06:38:10.600554 12492 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:45631 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:38:10.600638 12492 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.600710 12492 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:10.601029 12492 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.601107 12492 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
W20260810 06:38:10.601404 12492 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:38:10.719136 12497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:10.719481 12497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:10.719532 12497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:10.721621 12497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:10.721690 12497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:10.721743 12497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:38:10.724330 12497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:41769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=39509
--tserver_master_addrs=127.30.71.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:10.724741 12497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:10.725063 12497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:10.731801 12506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.731828 12504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.731801 12503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:10.732630 12497 server_base.cc:1034] running on GCE node
I20260810 06:38:10.732868 12497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:10.733181 12497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:10.734398 12497 hybrid_clock.cc:648] HybridClock initialized: now 1786343890734377 us; error 47 us; skew 500 ppm
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:10.738045 12497 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:38:10.738543 12497 webserver.cc:460] Webserver started at http://127.30.71.2:39509/ using document root <none> and password file <none>
I20260810 06:38:10.738795 12497 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:10.738901 12497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:10.769068 12497 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:38:10.769954 12515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.770134 12497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:10.770239 12497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "78653496534848b28d4a59d4f8322656"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "861c4af66af160ab7c8b854a5bb00e90"
server_key_iv: "625bad2264e7b49333825c052b966462"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:10.770766 12497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:10.813603 12497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:10.816257 12497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:10.816434 12497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:10.816716 12497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:10.817166 12497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:10.817279 12497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.817368 12497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:10.817436 12497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.826195 12497 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:41769
I20260810 06:38:10.826260 12642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:41769 every 8 connection(s)
I20260810 06:38:10.826812 12497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:10.836647 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 12497
I20260810 06:38:10.836984 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.71.20:49653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:35929
--local_ip_for_outbound_sockets=127.30.71.3
--tserver_master_addrs=127.30.71.62:45631
--webserver_port=41763
--webserver_interface=127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:10.839157 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.827733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:49147 (local address 127.30.71.62:45631)
0810 06:38:10.828199 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:10.828201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:10.828212 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:38:10.828799 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:10.828800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:10.828813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:10.828860 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:10.829721 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.830255 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.831352 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.831567 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.831642 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.835054 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:10.835056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:10.835056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:10.835064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:10.836755 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.837688 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.837689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.837690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.837784 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:10.838174 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:10.838175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:10.838176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:10.838366 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:10.838424 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:10.838903 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 06:38:10.839047 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439}
I20260810 06:38:10.840332 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:10.827717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45631 (local address 127.30.71.2:49147)
0810 06:38:10.827919 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:10.827974 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:10.828676 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:10.828951 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:10.828956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:10.829269 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:10.829593 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.829600 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.830360 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.830361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.831256 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:10.831259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:10.831680 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:10.831681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:10.831800 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:10.833363 (+ 1563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:10.833372 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:10.834894 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:10.837213 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.837222 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.837225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.837587 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.837889 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:10.837891 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:10.837891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:10.838066 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:10.838489 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:10.838499 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:10.838712 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 06:38:10.839809 (+ 1097us) client_negotiation.cc:240] Negotiation successful
0810 06:38:10.840030 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260810 06:38:10.841234 12643 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45631
I20260810 06:38:10.841411 12643 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.841692 12643 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
W20260810 06:38:10.842476 12643 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:45631 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:38:10.842631 12643 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.842749 12643 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:10.843194 12643 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:10.843326 12643 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
W20260810 06:38:10.843693 12643 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:38:10.955912 12646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:10.956311 12646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:10.956418 12646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:10.958300 12646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:10.958436 12646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:10.958531 12646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:38:10.960903 12646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.71.20:49653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:35929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=41763
--tserver_master_addrs=127.30.71.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:10.961175 12646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:10.961474 12646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:10.967033 12654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.967065 12656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:10.967077 12653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:10.967499 12646 server_base.cc:1034] running on GCE node
I20260810 06:38:10.967705 12646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:10.967998 12646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:10.969170 12646 hybrid_clock.cc:648] HybridClock initialized: now 1786343890969156 us; error 30 us; skew 500 ppm
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:10.975577 12646 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:38:10.976217 12646 webserver.cc:460] Webserver started at http://127.30.71.3:41763/ using document root <none> and password file <none>
I20260810 06:38:10.976501 12646 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:10.976588 12646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:38:10 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:10.991536 12646 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:38:10.992278 12666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:10.992450 12646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:10.992547 12646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "c9cb3a92ec614ff6882c8e532ef7525d"
format_stamp: "Formatted at 2026-08-10 06:38:09 on dist-test-slave-w206"
server_key: "90a993b1dbc7ed36439e0e799a678974"
server_key_iv: "71ae01ded22dcc328e4fcecd941a37af"
server_key_version: "kuduclusterkey@0"
I20260810 06:38:10.992944 12646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:11.007197 12646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:11.009843 12646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:11.010070 12646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:11.010385 12646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:11.010810 12646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:11.010902 12646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:11.010962 12646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:11.011044 12646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:11.017970 12646 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:35929
I20260810 06:38:11.018285 12789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:35929 every 8 connection(s)
I20260810 06:38:11.018538 12646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:11.026011 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 12646
Aug 10 06:38:11 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:11.030637 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.019426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:44355 (local address 127.30.71.62:45631)
0810 06:38:11.019483 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.019485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.019810 (+ 325us) server_negotiation.cc:407] Connection header received
0810 06:38:11.020591 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.020592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.020610 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.020686 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:11.021523 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.022017 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.023746 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.023947 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.024006 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.027223 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:11.027225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:11.027225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:11.027234 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:11.028875 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.029423 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.029424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.029425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.029503 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.029830 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.029832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.029832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.029998 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:11.030088 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.030449 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.030541 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:38:11.030853 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.019413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:45631 (local address 127.30.71.3:44355)
0810 06:38:11.019595 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:11.019671 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:11.020434 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:11.020796 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:11.020801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:11.021104 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:11.021410 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:11.021419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.022124 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.022125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:11.023631 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:11.023635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.024060 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.024062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:11.024222 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.025043 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:11.025055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:11.027108 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:11.028981 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:11.028989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:11.028992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:11.029324 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:11.029589 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:11.029590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:11.029591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:11.029742 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:11.030093 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:11.030101 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:11.030301 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 06:38:11.030423 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:38:11.030588 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260810 06:38:11.031477 12790 heartbeater.cc:347] Connected to a master server at 127.30.71.62:45631
I20260810 06:38:11.031605 12790 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:11.031840 12790 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
W20260810 06:38:11.032626 12790 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:45631 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:38:11.032719 12790 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:11.032789 12790 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:11.033231 12790 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:11.033337 12790 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
W20260810 06:38:11.033701 12790 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:38:11.037881 12794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:11.026672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.30.71.62:45631)
0810 06:38:11.026843 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:11.026846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:11.027554 (+ 708us) server_negotiation.cc:407] Connection header received
0810 06:38:11.027582 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:11.027583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:11.027597 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:11.027668 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:11.029154 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.030123 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.031372 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:11.032276 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:11.032351 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:11.034318 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:11.034319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:11.034320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:11.034327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:11.036245 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.036734 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.036736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.036736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.036811 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:11.037052 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:11.037054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:11.037054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:11.037206 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:11.037414 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:11.037517 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:38:11.037627 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
Aug 10 06:38:11 dist-test-slave-w206 krb5kdc[11202](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786343891, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:11.601641 12492 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:11.601828 12492 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:11.602248 12276 ts_manager.cc:194] Registered new tserver with Master: d6a5fe508d214b5294f4ed685585cdae (127.30.71.1:44707)
I20260810 06:38:11.603117 12276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:60041
I20260810 06:38:11.843945 12643 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:11.844133 12643 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:11.844481 12276 ts_manager.cc:194] Registered new tserver with Master: 78653496534848b28d4a59d4f8322656 (127.30.71.2:41769)
I20260810 06:38:11.844997 12276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:49147
Aug 10 06:38:12 dist-test-slave-w206 krb5kdc[11202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343891, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for HTTP/127.30.71.20@KRBTEST.COM
I20260810 06:38:12.033917 12790 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:12.034091 12790 heartbeater.cc:511] Master 127.30.71.62:45631 requested a full tablet report, sending...
I20260810 06:38:12.034487 12276 ts_manager.cc:194] Registered new tserver with Master: c9cb3a92ec614ff6882c8e532ef7525d (127.30.71.3:35929)
I20260810 06:38:12.035034 12276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:44355
I20260810 06:38:12.038614 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:12.049968 12814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.039690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35996 (local address 127.30.71.62:45631)
0810 06:38:12.040325 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.040327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.040496 (+ 169us) server_negotiation.cc:407] Connection header received
0810 06:38:12.040522 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.040523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.040536 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.041799 (+ 1263us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:12.041844 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.043295 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.044292 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.044564 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.044627 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.046262 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:12.046264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:12.046264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:12.046272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:12.048223 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:12.048652 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:12.048653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:12.048654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:12.048729 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:12.048944 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:12.048945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:12.048946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:12.049093 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:12.049197 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.049766 (+ 569us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.049863 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":94,"threads_started":1}
I20260810 06:38:12.052474 12276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:12.055069 12276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:12.068779 12824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.062363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:35929 (local address 127.0.0.1:45636)
0810 06:38:12.063552 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.063560 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.063601 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.063895 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.063898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.063900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.063972 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.063974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.065610 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.065611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.067351 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.067354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.068449 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.068450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.068534 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.068548 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.068619 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.068684 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"thread_start_us":87,"threads_started":1}
I20260810 06:38:12.069296 12828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.062632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.30.71.3:35929)
0810 06:38:12.063384 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.063389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.063660 (+ 271us) server_negotiation.cc:407] Connection header received
0810 06:38:12.063702 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.063705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.063722 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.063795 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.064053 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.065511 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.067440 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.068236 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.068327 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.068402 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.069142 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.069190 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":64,"threads_started":1}
I20260810 06:38:12.070418 12720 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a (DEFAULT_TABLE table=test-table [id=3c645062ec4340c7a3c1d0024357ec38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:12.070830 12720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:12.072248 12822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.062168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41769 (local address 127.0.0.1:33328)
0810 06:38:12.063435 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.063444 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.063483 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.067410 (+ 3927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.067412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.067414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.067477 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.067479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.068741 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.068742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.069883 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.069885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.071985 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.071986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.072063 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.072069 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.072127 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.072176 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1253,"thread_start_us":83,"threads_started":1}
I20260810 06:38:12.074908 12829 tablet_bootstrap.cc:492] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Bootstrap starting.
I20260810 06:38:12.075713 12829 tablet_bootstrap.cc:654] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:12.075945 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.062275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.30.71.2:41769)
0810 06:38:12.067130 (+ 4855us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.067136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.067150 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:38:12.067191 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.067194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.067211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.067311 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.067556 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.068628 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.071047 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.071781 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.071870 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.071947 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.075793 (+ 3846us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.075854 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4809,"thread_start_us":75,"threads_started":1}
I20260810 06:38:12.076133 12829 log.cc:824] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:12.077018 12829 tablet_bootstrap.cc:492] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: No bootstrap required, opened a new log
I20260810 06:38:12.077083 12829 ts_tablet_manager.cc:1397] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:12.077201 12566 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a (DEFAULT_TABLE table=test-table [id=3c645062ec4340c7a3c1d0024357ec38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:12.077579 12566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:12.079571 12829 raft_consensus.cc:348] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.079734 12829 raft_consensus.cc:374] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:12.079783 12829 raft_consensus.cc:729] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cb3a92ec614ff6882c8e532ef7525d, State: Initialized, Role: FOLLOWER
I20260810 06:38:12.080497 12829 consensus_queue.cc:260] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.080922 12829 ts_tablet_manager.cc:1428] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:38:12.081089 12790 heartbeater.cc:503] Master 127.30.71.62:45631 was elected leader, sending a full tablet report...
I20260810 06:38:12.084885 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.061914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44707 (local address 127.0.0.1:54968)
0810 06:38:12.063126 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.063172 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.063247 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.076213 (+ 12966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.076215 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.076219 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.076272 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.076275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.082529 (+ 6254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.082530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.083671 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.083675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.084637 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.084638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.084724 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.084729 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.084761 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.084807 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"thread_start_us":79,"threads_started":1}
I20260810 06:38:12.085479 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.061998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54968 (local address 127.30.71.1:44707)
0810 06:38:12.075572 (+ 13574us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.075577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.075591 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:38:12.075626 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.075628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.075654 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.076366 (+ 712us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.076457 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.082432 (+ 5975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.083786 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.084523 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.085238 (+ 715us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.085316 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.085346 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.085393 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13531,"thread_start_us":63,"threads_started":1}
I20260810 06:38:12.085721 12832 tablet_bootstrap.cc:492] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Bootstrap starting.
I20260810 06:38:12.086397 12832 tablet_bootstrap.cc:654] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:12.086333 12420 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a (DEFAULT_TABLE table=test-table [id=3c645062ec4340c7a3c1d0024357ec38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:12.086670 12420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:12.086759 12832 log.cc:824] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:12.087651 12832 tablet_bootstrap.cc:492] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: No bootstrap required, opened a new log
I20260810 06:38:12.087776 12832 ts_tablet_manager.cc:1397] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:38:12.090065 12832 raft_consensus.cc:348] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.090310 12832 raft_consensus.cc:374] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:12.090390 12832 raft_consensus.cc:729] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78653496534848b28d4a59d4f8322656, State: Initialized, Role: FOLLOWER
I20260810 06:38:12.090582 12832 consensus_queue.cc:260] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.090924 12832 ts_tablet_manager.cc:1428] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:38:12.091130 12643 heartbeater.cc:503] Master 127.30.71.62:45631 was elected leader, sending a full tablet report...
I20260810 06:38:12.092411 12834 tablet_bootstrap.cc:492] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae: Bootstrap starting.
I20260810 06:38:12.093149 12834 tablet_bootstrap.cc:654] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:12.093544 12834 log.cc:824] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:12.094508 12834 tablet_bootstrap.cc:492] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae: No bootstrap required, opened a new log
I20260810 06:38:12.094578 12834 ts_tablet_manager.cc:1397] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:12.096968 12834 raft_consensus.cc:348] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.097137 12834 raft_consensus.cc:374] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:12.097190 12834 raft_consensus.cc:729] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a5fe508d214b5294f4ed685585cdae, State: Initialized, Role: FOLLOWER
I20260810 06:38:12.097640 12834 consensus_queue.cc:260] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.098002 12834 ts_tablet_manager.cc:1428] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:38:12.098194 12492 heartbeater.cc:503] Master 127.30.71.62:45631 was elected leader, sending a full tablet report...
I20260810 06:38:12.211921 12838 raft_consensus.cc:482] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:12.212093 12838 raft_consensus.cc:504] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.212502 12838 leader_election.cc:288] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9cb3a92ec614ff6882c8e532ef7525d (127.30.71.3:35929), 78653496534848b28d4a59d4f8322656 (127.30.71.2:41769)
I20260810 06:38:12.217264 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.213017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:38869 (local address 127.30.71.2:41769)
0810 06:38:12.213065 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.213068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.213639 (+ 571us) server_negotiation.cc:407] Connection header received
0810 06:38:12.213750 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.213751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.213763 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.213794 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.214039 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.214788 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.216307 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.216853 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.216904 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.216916 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.217131 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.217174 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:38:12.217568 12839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.212769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41769 (local address 127.30.71.1:38869)
0810 06:38:12.213567 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.213578 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.213669 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.213867 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.213868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.213870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.213964 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.213966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.214900 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.214901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.216174 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.216178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.216980 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.216981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.217053 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.217057 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.217417 (+ 360us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.217466 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":111,"threads_started":1}
I20260810 06:38:12.217931 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac7dfb19cf24c75aaad1a04a2101d4a" candidate_uuid: "d6a5fe508d214b5294f4ed685585cdae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78653496534848b28d4a59d4f8322656" is_pre_election: true
I20260810 06:38:12.218146 12595 raft_consensus.cc:2454] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a5fe508d214b5294f4ed685585cdae in term 0.
I20260810 06:38:12.218461 12371 leader_election.cc:302] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78653496534848b28d4a59d4f8322656, d6a5fe508d214b5294f4ed685585cdae; no voters:
I20260810 06:38:12.218627 12838 raft_consensus.cc:2781] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:12.218703 12838 raft_consensus.cc:482] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:12.218736 12838 raft_consensus.cc:3037] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:12.219806 12838 raft_consensus.cc:504] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.220006 12838 leader_election.cc:288] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [CANDIDATE]: Term 1 election: Requested vote from peers c9cb3a92ec614ff6882c8e532ef7525d (127.30.71.3:35929), 78653496534848b28d4a59d4f8322656 (127.30.71.2:41769)
I20260810 06:38:12.220253 12595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac7dfb19cf24c75aaad1a04a2101d4a" candidate_uuid: "d6a5fe508d214b5294f4ed685585cdae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78653496534848b28d4a59d4f8322656"
I20260810 06:38:12.220350 12595 raft_consensus.cc:3037] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:12.221235 12828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.213460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:51487 (local address 127.30.71.3:35929)
0810 06:38:12.213504 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.213506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.217851 (+ 4345us) server_negotiation.cc:407] Connection header received
0810 06:38:12.217954 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.217955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.217968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.217997 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:12.218151 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.218837 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.220264 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.220843 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.220895 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.220906 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.221102 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.221147 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:38:12.221320 12595 raft_consensus.cc:2454] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a5fe508d214b5294f4ed685585cdae in term 1.
I20260810 06:38:12.221467 12839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.213189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:35929 (local address 127.30.71.1:51487)
0810 06:38:12.217797 (+ 4608us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.217808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:12.217882 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:12.218054 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:12.218055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:12.218057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:12.218087 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.218089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.218952 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.218953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.220136 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:12.220139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.220966 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.220967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:12.221025 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.221027 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:38:12.221362 (+ 335us) client_negotiation.cc:240] Negotiation successful
0810 06:38:12.221403 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4594,"thread_start_us":163,"threads_started":1}
I20260810 06:38:12.221515 12371 leader_election.cc:302] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78653496534848b28d4a59d4f8322656, d6a5fe508d214b5294f4ed685585cdae; no voters:
I20260810 06:38:12.221604 12838 raft_consensus.cc:2781] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:12.221797 12838 raft_consensus.cc:686] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [term 1 LEADER]: Becoming Leader. State: Replica: d6a5fe508d214b5294f4ed685585cdae, State: Running, Role: LEADER
I20260810 06:38:12.221783 12723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac7dfb19cf24c75aaad1a04a2101d4a" candidate_uuid: "d6a5fe508d214b5294f4ed685585cdae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" is_pre_election: true
I20260810 06:38:12.221902 12838 consensus_queue.cc:237] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } }
I20260810 06:38:12.221939 12723 raft_consensus.cc:2454] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a5fe508d214b5294f4ed685585cdae in term 0.
I20260810 06:38:12.222060 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac7dfb19cf24c75aaad1a04a2101d4a" candidate_uuid: "d6a5fe508d214b5294f4ed685585cdae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cb3a92ec614ff6882c8e532ef7525d"
I20260810 06:38:12.222121 12721 raft_consensus.cc:3037] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:12.223030 12721 raft_consensus.cc:2454] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a5fe508d214b5294f4ed685585cdae in term 1.
I20260810 06:38:12.222921 12275 catalog_manager.cc:5495] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a5fe508d214b5294f4ed685585cdae (127.30.71.1). New cstate: current_term: 1 leader_uuid: "d6a5fe508d214b5294f4ed685585cdae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5fe508d214b5294f4ed685585cdae" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:12.250783 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.247424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.30.71.1:44707)
0810 06:38:12.247470 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:12.247473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:12.247484 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:38:12.247533 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:12.247534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:12.247559 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:12.247669 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:12.247827 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.248542 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.249652 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:12.249964 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:12.250026 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:12.250239 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:12.250383 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:12.250462 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:12.250643 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 06:38:12.250666 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:38:12.253582 12595 raft_consensus.cc:1261] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 1 FOLLOWER]: Refusing update from remote peer d6a5fe508d214b5294f4ed685585cdae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:12.253580 12721 raft_consensus.cc:1261] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 1 FOLLOWER]: Refusing update from remote peer d6a5fe508d214b5294f4ed685585cdae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:12.254117 12838 consensus_queue.cc:1035] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [LEADER]: Connected to new peer: Peer: permanent_uuid: "78653496534848b28d4a59d4f8322656" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:12.254439 12838 consensus_queue.cc:1035] T 8ac7dfb19cf24c75aaad1a04a2101d4a P d6a5fe508d214b5294f4ed685585cdae [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cb3a92ec614ff6882c8e532ef7525d" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 35929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:12.261507 12844 mvcc.cc:205] Tried to move back new op lower bound from 7316864582665428992 to 7316864582541656064. Current Snapshot: MvccSnapshot[applied={T|T < 7316864582665428992}]
I20260810 06:38:12.261723 12845 mvcc.cc:205] Tried to move back new op lower bound from 7316864582665428992 to 7316864582541656064. Current Snapshot: MvccSnapshot[applied={T|T < 7316864582665428992}]
I20260810 06:38:12.265185 12276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:12.265302 12276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:12.266090 12849 mvcc.cc:205] Tried to move back new op lower bound from 7316864582665428992 to 7316864582541656064. Current Snapshot: MvccSnapshot[applied={T|T < 7316864582665428992}]
I20260810 06:38:12.266145 12276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f5ee9d93db84b93b3130cd6929463cb: Sending DeleteTablet for 3 replicas of tablet 8ac7dfb19cf24c75aaad1a04a2101d4a
I20260810 06:38:12.266508 12420 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:12 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:54968
I20260810 06:38:12.266695 12566 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:12 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:33328
I20260810 06:38:12.267036 12720 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:12 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:45636
I20260810 06:38:12.267536 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 12316
I20260810 06:38:12.279148 12857 tablet_replica.cc:330] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: stopping tablet replica
I20260810 06:38:12.279335 12857 raft_consensus.cc:2229] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:12.279452 12857 raft_consensus.cc:2258] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:12.280184 12857 ts_tablet_manager.cc:1905] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:12.282671 12857 ts_tablet_manager.cc:1918] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:12.282785 12857 log.cc:1196] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/8ac7dfb19cf24c75aaad1a04a2101d4a
I20260810 06:38:12.283012 12856 tablet_replica.cc:330] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: stopping tablet replica
I20260810 06:38:12.283182 12857 ts_tablet_manager.cc:1939] T 8ac7dfb19cf24c75aaad1a04a2101d4a P 78653496534848b28d4a59d4f8322656: Deleting consensus metadata
I20260810 06:38:12.283216 12856 raft_consensus.cc:2229] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:12.283505 12856 raft_consensus.cc:2258] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:12.283815 12259 catalog_manager.cc:4841] TS 78653496534848b28d4a59d4f8322656 (127.30.71.2:41769): tablet 8ac7dfb19cf24c75aaad1a04a2101d4a (table test-table [id=3c645062ec4340c7a3c1d0024357ec38]) successfully deleted
I20260810 06:38:12.284194 12856 ts_tablet_manager.cc:1905] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:12.285929 12856 ts_tablet_manager.cc:1918] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:12.286073 12856 log.cc:1196] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/8ac7dfb19cf24c75aaad1a04a2101d4a
I20260810 06:38:12.286422 12856 ts_tablet_manager.cc:1939] T 8ac7dfb19cf24c75aaad1a04a2101d4a P c9cb3a92ec614ff6882c8e532ef7525d: Deleting consensus metadata
I20260810 06:38:12.286947 12261 catalog_manager.cc:4841] TS c9cb3a92ec614ff6882c8e532ef7525d (127.30.71.3:35929): tablet 8ac7dfb19cf24c75aaad1a04a2101d4a (table test-table [id=3c645062ec4340c7a3c1d0024357ec38]) successfully deleted
W20260810 06:38:12.291062 12262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:44707 (error 108)
I20260810 06:38:12.291538 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 12497
I20260810 06:38:12.291699 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:12.291563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44707 (local address 127.0.0.1:54986)
0810 06:38:12.291623 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:12.291658 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:38:12.293712 12262 catalog_manager.cc:4584] TS d6a5fe508d214b5294f4ed685585cdae (127.30.71.1:44707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ac7dfb19cf24c75aaad1a04a2101d4a: Network error: Client connection negotiation failed: client connection to 127.30.71.1:44707: connect: Connection refused (error 111)
I20260810 06:38:12.302347 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 12646
I20260810 06:38:12.311018 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 12242
I20260810 06:38:12.318272 31004 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:38:12.663762 31004 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:38:12.663882 31004 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:38:13.009084 31004 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:38:13.009 UTC [11232] LOG: received smart shutdown request
2026-08-10 06:38:13.014 UTC [11232] LOG: background worker "logical replication launcher" (PID 11243) exited with exit code 1
2026-08-10 06:38:13.014 UTC [11238] LOG: shutting down
2026-08-10 06:38:13.201 UTC [11232] LOG: database system is shut down
2026-08-10T06:38:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49124 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:13 dist-test-slave-w206 krb5kdc[12867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:13 dist-test-slave-w206 krb5kdc[12867](info): set up 2 sockets
Aug 10 06:38:13 dist-test-slave-w206 krb5kdc[12867](info): commencing operation
krb5kdc: starting...
W20260810 06:38:14.444317 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:38:14 dist-test-slave-w206 krb5kdc[12867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:38:14.583115 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:46203
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:46203,127.30.71.61:39575,127.30.71.60:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:14.667544 12881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:14.667783 12881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:14.667809 12881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:14.669075 12881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:14.669131 12881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:14.669147 12881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:14.669159 12881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:14.669173 12881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:14.670706 12881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:46203,127.30.71.61:39575,127.30.71.60:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:46203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:14.670922 12881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:14.671226 12881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:14.676534 12887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:14.676537 12889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:14.676805 12881 server_base.cc:1034] running on GCE node
W20260810 06:38:14.676548 12886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:14.677162 12881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:14.677500 12881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:14.678686 12881 hybrid_clock.cc:648] HybridClock initialized: now 1786343894678658 us; error 44 us; skew 500 ppm
Aug 10 06:38:14 dist-test-slave-w206 krb5kdc[12867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:14.681747 12881 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:14.682273 12881 webserver.cc:460] Webserver started at http://127.30.71.62:37375/ using document root <none> and password file <none>
I20260810 06:38:14.682544 12881 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:14.682623 12881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:14.682794 12881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:14.684808 12881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "21a0f3e4cf084a1192db2966a0851b91"
format_stamp: "Formatted at 2026-08-10 06:38:14 on dist-test-slave-w206"
server_key: "eeb438dbdfa17c004e8fc872c7db1494"
server_key_iv: "fecb8c931b9f35cd0f7c6f263a68ea95"
server_key_version: "encryptionkey@0"
I20260810 06:38:14.685354 12881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "21a0f3e4cf084a1192db2966a0851b91"
format_stamp: "Formatted at 2026-08-10 06:38:14 on dist-test-slave-w206"
server_key: "eeb438dbdfa17c004e8fc872c7db1494"
server_key_iv: "fecb8c931b9f35cd0f7c6f263a68ea95"
server_key_version: "encryptionkey@0"
I20260810 06:38:14.687495 12881 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260810 06:38:14.688568 12896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:14.688794 12881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:14.688905 12881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "21a0f3e4cf084a1192db2966a0851b91"
format_stamp: "Formatted at 2026-08-10 06:38:14 on dist-test-slave-w206"
server_key: "eeb438dbdfa17c004e8fc872c7db1494"
server_key_iv: "fecb8c931b9f35cd0f7c6f263a68ea95"
server_key_version: "encryptionkey@0"
I20260810 06:38:14.688992 12881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:14.707839 12881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:14.710060 12881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:14.710186 12881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:14.714298 12881 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:46203
I20260810 06:38:14.714354 12959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:46203 every 8 connection(s)
I20260810 06:38:14.714748 12881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:14.715827 12962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:14.717788 12962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:38:14.718924 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 12881
I20260810 06:38:14.719095 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:38:14.719270 31004 external_mini_cluster.cc:1428] Setting key c49e12f1f58b562a64a5e258edf13ebe
Aug 10 06:38:14 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.61@KRBTEST.COM; defaulting to no policy
I20260810 06:38:14.728708 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.718532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:46203 (local address 127.0.0.1:37648)
0810 06:38:14.718802 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.718851 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:14.719525 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:14.719846 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:14.719848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:14.720165 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:14.720323 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.720328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.721372 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.721372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.722061 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.722063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.722591 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.722592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.722655 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.723312 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:14.723321 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:14.724792 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:14.727005 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.727012 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.727015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.727240 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.727520 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.727521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.727522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.727621 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.728140 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:14.728147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:14.728248 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 06:38:14.728336 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:38:14.728479 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":95,"threads_started":1}
I20260810 06:38:14.728713 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.718604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.30.71.62:46203)
0810 06:38:14.718814 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:14.718824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:14.718962 (+ 138us) server_negotiation.cc:407] Connection header received
0810 06:38:14.719697 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:14.719699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:14.719717 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:14.719838 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:14.720440 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.721257 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.722168 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.722490 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.722624 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.724915 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:14.724922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:14.724922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:14.724928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:14.726844 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.727338 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.727341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.727342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.727439 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.727704 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.727705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.727706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.728052 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:14.728149 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:14.728400 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 06:38:14.728549 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.61@KRBTEST.COM" created.
I20260810 06:38:14.729688 12962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:38:14.730095 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.729956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59254)
0810 06:38:14.730010 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.730063 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:38:14.730252 12899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
I20260810 06:38:14.730731 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.730633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59256)
0810 06:38:14.730682 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.730708 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:38:14.730858 12962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:39575: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab.
Entry for principal kudu/127.30.71.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab.
I20260810 06:38:14.747035 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:39575
--webserver_interface=127.30.71.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab
--principal=kudu/127.30.71.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:46203,127.30.71.61:39575,127.30.71.60:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:14.780120 12962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } attempt: 1
I20260810 06:38:14.780707 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.780548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59270)
0810 06:38:14.780615 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.780664 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:38:14.781088 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.780986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59276)
0810 06:38:14.781037 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.781068 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:38:14.781225 12962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:39575: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
W20260810 06:38:14.832460 12970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:14.832683 12970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:14.832707 12970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:14.833911 12970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:14.833961 12970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:14.833974 12970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:14.833987 12970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:14.834000 12970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:14.835644 12970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:46203,127.30.71.61:39575,127.30.71.60:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab
--principal=kudu/127.30.71.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.71.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:14.835915 12970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:14.836169 12970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:14.841502 12976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:14.841512 12980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:14.841508 12977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:14.841813 12970 server_base.cc:1034] running on GCE node
I20260810 06:38:14.842083 12970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:14.842342 12970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:14.843477 12970 hybrid_clock.cc:648] HybridClock initialized: now 1786343894843446 us; error 46 us; skew 500 ppm
Aug 10 06:38:14 dist-test-slave-w206 krb5kdc[12867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:14.846215 12970 init.cc:377] Logged in from keytab as kudu/127.30.71.61@KRBTEST.COM (short username kudu)
I20260810 06:38:14.846742 12970 webserver.cc:460] Webserver started at http://127.30.71.61:40053/ using document root <none> and password file <none>
I20260810 06:38:14.847072 12970 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:14.847151 12970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:14.847302 12970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:14.848428 12970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data/instance:
uuid: "99c53355b5a34b2f9afbcea33cac44ae"
format_stamp: "Formatted at 2026-08-10 06:38:14 on dist-test-slave-w206"
server_key: "52d37a4b2efac271d50c66f43ee3dffd"
server_key_iv: "80aa705731092eeb9ea202e05db9106d"
server_key_version: "encryptionkey@0"
I20260810 06:38:14.848824 12970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/wal/instance:
uuid: "99c53355b5a34b2f9afbcea33cac44ae"
format_stamp: "Formatted at 2026-08-10 06:38:14 on dist-test-slave-w206"
server_key: "52d37a4b2efac271d50c66f43ee3dffd"
server_key_iv: "80aa705731092eeb9ea202e05db9106d"
server_key_version: "encryptionkey@0"
I20260810 06:38:14.849398 12962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } attempt: 2
I20260810 06:38:14.849813 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.849657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59284)
0810 06:38:14.849729 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.849771 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 06:38:14.850219 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.850089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59298)
0810 06:38:14.850162 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.850194 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 06:38:14.850322 12962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:39575: Network error: Client connection negotiation failed: client connection to 127.30.71.61:39575: connect: Connection refused (error 111)
I20260810 06:38:14.850452 12970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:38:14.851423 12987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:14.851625 12970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:14.851729 12970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/wal
uuid: "99c53355b5a34b2f9afbcea33cac44ae"
format_stamp: "Formatted at 2026-08-10 06:38:14 on dist-test-slave-w206"
server_key: "52d37a4b2efac271d50c66f43ee3dffd"
server_key_iv: "80aa705731092eeb9ea202e05db9106d"
server_key_version: "encryptionkey@0"
I20260810 06:38:14.851794 12970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:14.865063 12970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:14.867337 12970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:14.867476 12970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:14.871865 12970 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.61:39575
I20260810 06:38:14.871937 13048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.61:39575 every 8 connection(s)
I20260810 06:38:14.872380 12970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
I20260810 06:38:14.873311 13049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:14.874840 13049 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:14 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:14.882622 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 12970
I20260810 06:38:14.882781 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-1/wal/instance
I20260810 06:38:14.883138 31004 external_mini_cluster.cc:1428] Setting key 78f9506104d0e85bff264cde14c9f5d7
I20260810 06:38:14.884294 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.875628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.30.71.62:46203)
0810 06:38:14.875695 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:14.875697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:14.875907 (+ 210us) server_negotiation.cc:407] Connection header received
0810 06:38:14.876348 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:14.876349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:14.876363 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:14.876425 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:14.877049 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.877538 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.878532 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.878732 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.878789 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.881076 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:14.881077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:14.881078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:14.881084 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:14.882333 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.882935 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.882936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.882937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.883067 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.883486 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.883487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.883487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.883654 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:14.883776 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:14.884107 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 06:38:14.884204 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:38:14.884487 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.875588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:46203 (local address 127.0.0.1:37664)
0810 06:38:14.875770 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.875814 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:14.876265 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:14.876507 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:14.876510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:14.876764 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:14.876965 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.876968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.877623 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.877623 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.878452 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.878454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.878822 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.878823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.878942 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.879598 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:14.879608 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:14.880988 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:14.882450 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.882459 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.882466 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.882773 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.883179 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.883180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.883180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.883365 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.883769 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:14.883777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:14.883958 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 06:38:14.884072 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 06:38:14.884239 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260810 06:38:14.885061 13049 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:14 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.60@KRBTEST.COM; defaulting to no policy
I20260810 06:38:14.892856 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.885355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59308)
0810 06:38:14.885397 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.885408 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:14.885449 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:14.885698 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:14.885699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:14.885847 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:14.885896 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.885898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.886758 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.886759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.887439 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.887441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.887941 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.887942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.888013 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.888481 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:14.888486 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:14.889680 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:14.891804 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.891805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.891805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.891988 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.892188 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.892188 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.892189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.892243 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.892577 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:14.892578 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:14.892631 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:38:14.892693 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 06:38:14.892781 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 06:38:14.893054 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.885396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.30.71.61:39575)
0810 06:38:14.885546 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:14.885550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:14.885567 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:38:14.885602 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:14.885604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:14.885615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:14.885698 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:14.885982 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.886666 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.887524 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.887844 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.887941 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.889773 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:14.889783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:14.889783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:14.889792 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:14.891715 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.892068 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.892071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.892071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.892127 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.892298 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.892298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.892298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.892505 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:14.892572 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:14.892731 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 06:38:14.892835 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260810 06:38:14.893672 13049 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:38:14.894062 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.893937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44872)
0810 06:38:14.893987 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.894041 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:38:14.894230 12993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.60@KRBTEST.COM" created.
I20260810 06:38:14.894754 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.894651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44882)
0810 06:38:14.894697 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.894723 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 06:38:14.894917 13049 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:36929: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab.
Entry for principal kudu/127.30.71.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab.
I20260810 06:38:14.911226 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:36929
--webserver_interface=127.30.71.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab
--principal=kudu/127.30.71.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:46203,127.30.71.61:39575,127.30.71.60:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:14.941489 12962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } attempt: 3
I20260810 06:38:14.944186 13049 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } attempt: 1
I20260810 06:38:14.944605 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.944479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44884)
0810 06:38:14.944520 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.944561 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 06:38:14.945137 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.944911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44896)
0810 06:38:14.945066 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.945099 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
W20260810 06:38:14.945271 13049 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:36929: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Aug 10 06:38:14 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:38:14.949307 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.941924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59310)
0810 06:38:14.942020 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.942040 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:14.942134 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:14.942389 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:14.942390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:14.942577 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:14.942644 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.942645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.943496 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.943497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.943998 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:14.944000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.944448 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.944449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:14.944532 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.945073 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:14.945079 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:14.946427 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:14.948134 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.948135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.948135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.948326 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.948593 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:14.948594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:14.948594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:14.948663 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:14.948955 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:14.948957 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:14.949008 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:38:14.949097 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:38:14.949209 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:38:14.949307 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.942049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59310 (local address 127.30.71.61:39575)
0810 06:38:14.942116 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:14.942118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:14.942130 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:38:14.942288 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:14.942290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:14.942305 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:14.942381 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:14.942736 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.943405 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.944103 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:14.944372 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:14.944440 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:14.946519 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:14.946521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:14.946521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:14.946538 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:14.948034 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.948464 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.948465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.948465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.948520 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:14.948729 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:14.948730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:14.948731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:14.948851 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:14.948962 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:14.949122 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 06:38:14.949211 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:38:14.949738 12962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:38:14.950106 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.950001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44912)
0810 06:38:14.950051 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.950080 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 06:38:14.950641 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.950467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44922)
0810 06:38:14.950586 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.950618 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260810 06:38:14.950812 12962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:36929: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
I20260810 06:38:14.981997 12962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } attempt: 1
I20260810 06:38:14.982630 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.982465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44934)
0810 06:38:14.982555 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.982595 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:38:14.983063 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:14.982928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44942)
0810 06:38:14.983004 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:14.983034 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:38:14.983170 12962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:36929: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
W20260810 06:38:14.996469 13056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:14.996713 13056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:14.996752 13056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:14.997956 13056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:14.998030 13056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:14.998054 13056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:14.998076 13056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:14.998106 13056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:14.999716 13056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:46203,127.30.71.61:39575,127.30.71.60:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab
--principal=kudu/127.30.71.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:36929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.71.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:15.000036 13056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:15.000296 13056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:15.005332 13062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:15.005342 13066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:15.005336 13064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:15.005650 13056 server_base.cc:1034] running on GCE node
I20260810 06:38:15.005892 13056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:15.006163 13056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:15.007333 13056 hybrid_clock.cc:648] HybridClock initialized: now 1786343895007319 us; error 32 us; skew 500 ppm
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:15.010141 13056 init.cc:377] Logged in from keytab as kudu/127.30.71.60@KRBTEST.COM (short username kudu)
I20260810 06:38:15.010609 13056 webserver.cc:460] Webserver started at http://127.30.71.60:41169/ using document root <none> and password file <none>
I20260810 06:38:15.010867 13056 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:15.010939 13056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:15.011133 13056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:15.012256 13056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data/instance:
uuid: "bf4aedfe97b4476bbfa13a6eac21cdad"
format_stamp: "Formatted at 2026-08-10 06:38:15 on dist-test-slave-w206"
server_key: "ffeaa338fe1cd175f5d5c0b7682d0fbe"
server_key_iv: "20a7c4302da1c930dd940786dff3483d"
server_key_version: "encryptionkey@0"
I20260810 06:38:15.012641 13056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/wal/instance:
uuid: "bf4aedfe97b4476bbfa13a6eac21cdad"
format_stamp: "Formatted at 2026-08-10 06:38:15 on dist-test-slave-w206"
server_key: "ffeaa338fe1cd175f5d5c0b7682d0fbe"
server_key_iv: "20a7c4302da1c930dd940786dff3483d"
server_key_version: "encryptionkey@0"
I20260810 06:38:15.013441 13049 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } attempt: 2
I20260810 06:38:15.013907 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.013772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44952)
0810 06:38:15.013838 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:15.013878 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:38:15.014211 13056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:38:15.014312 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.014206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44962)
0810 06:38:15.014258 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:15.014286 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:38:15.014482 13049 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:36929: Network error: Client connection negotiation failed: client connection to 127.30.71.60:36929: connect: Connection refused (error 111)
I20260810 06:38:15.015205 13073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:15.015388 13056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:15.015493 13056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/wal
uuid: "bf4aedfe97b4476bbfa13a6eac21cdad"
format_stamp: "Formatted at 2026-08-10 06:38:15 on dist-test-slave-w206"
server_key: "ffeaa338fe1cd175f5d5c0b7682d0fbe"
server_key_iv: "20a7c4302da1c930dd940786dff3483d"
server_key_version: "encryptionkey@0"
I20260810 06:38:15.015586 13056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:15.036861 13056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:15.038933 13056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:15.039176 13056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:15.043519 13056 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.60:36929
I20260810 06:38:15.043560 13133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.60:36929 every 8 connection(s)
I20260810 06:38:15.043962 13056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
I20260810 06:38:15.044902 13134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:15.046504 13134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:38:15.046953 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 13056
I20260810 06:38:15.047127 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786343734291041-31004-0/minicluster-data/master-2/wal/instance
I20260810 06:38:15.047358 31004 external_mini_cluster.cc:1428] Setting key d5c08912d436fb5fdfffea9d42072594
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
I20260810 06:38:15.057843 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.047355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.30.71.62:46203)
0810 06:38:15.047445 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.047448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.047708 (+ 260us) server_negotiation.cc:407] Connection header received
0810 06:38:15.048362 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.048363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.048375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.048437 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.049197 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.049956 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.051295 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.051631 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.051704 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.054276 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.054278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.054278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.054287 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.056212 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.056764 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.056765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.056766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.056819 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.057135 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.057136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.057136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.057269 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.057346 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.057651 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.057744 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:38:15.058065 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.047339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:46203 (local address 127.0.0.1:37674)
0810 06:38:15.047536 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:15.047580 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:15.048259 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:15.048458 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:15.048461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:15.048859 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:15.049103 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.049109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.050081 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.050081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.051198 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.051202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.051715 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.051715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.051814 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.052422 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:15.052434 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:15.054181 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:15.056296 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.056302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.056304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.056685 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.056932 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.056933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.056933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.057060 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.057348 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:15.057356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:15.057513 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 06:38:15.057627 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 06:38:15.057788 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260810 06:38:15.058271 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.047740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.30.71.60:36929)
0810 06:38:15.047907 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.047913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.048010 (+ 97us) server_negotiation.cc:407] Connection header received
0810 06:38:15.048272 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.048279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.048320 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.048433 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.049138 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.050312 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.051338 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.051815 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.051866 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.054948 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.054960 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.054961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.054967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.056810 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.057165 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.057168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.057169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.057228 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.057432 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.057433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.057433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.057581 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.057737 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.057824 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.057912 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260810 06:38:15.058310 12962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } attempt: 2
I20260810 06:38:15.058645 13134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:38:15.067734 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.058540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44980)
0810 06:38:15.058617 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:15.058628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:15.058679 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:15.058823 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:15.058824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:15.059004 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:15.059056 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.059058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.059965 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.059967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.060710 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.060712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.061223 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.061224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.061283 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.061746 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:15.061751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:15.064505 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:15.066382 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.066383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.066384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.066662 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.066909 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.066910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.066910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.067005 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.067346 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:15.067348 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:15.067404 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 06:38:15.067471 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:38:15.067588 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 06:38:15.067737 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.058580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.30.71.60:36929)
0810 06:38:15.058617 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.058619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.058668 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:38:15.058739 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.058740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.058749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.058821 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.059135 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.059855 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.060792 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.061117 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.061172 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.064602 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.064603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.064603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.064609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.066276 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.066747 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.066756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.066756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.066824 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.067082 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.067083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.067084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.067258 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.067332 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.067534 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.067622 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21,"thread_start_us":127,"threads_started":1}
I20260810 06:38:15.068291 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.059006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59322 (local address 127.30.71.61:39575)
0810 06:38:15.059160 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.059163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.059172 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:38:15.059195 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.059196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.059206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.059258 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.059729 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.060435 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.061330 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.062125 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.062176 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.064596 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.064598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.064598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.064606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.066997 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.067420 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.067422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.067422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.067486 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.067689 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.067690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.067690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.067833 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.067886 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.068121 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.068205 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260810 06:38:15.068356 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.058922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:39575 (local address 127.0.0.1:59322)
0810 06:38:15.058962 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:15.058992 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:15.059025 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:15.059418 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:15.059419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:15.059580 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:15.059644 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.059646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.060523 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.060524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.061241 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.061242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.062263 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.062264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.062352 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.062790 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:15.062795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:15.064505 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:15.067091 (+ 2586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.067092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.067092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.067326 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.067563 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.067564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.067564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.067620 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.067942 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:15.067943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:15.067993 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 06:38:15.068059 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:38:15.068147 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 06:38:15.068919 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.059402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.30.71.62:46203)
0810 06:38:15.059436 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.059438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.059445 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:38:15.059495 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.059496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.059504 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.059555 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.060071 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.060521 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.061199 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.061398 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.061479 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.064109 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.064111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.064111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.064117 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.067269 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.067692 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.067693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.067694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.067747 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.067980 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.067981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.067982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.068125 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.068322 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.068753 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.068834 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:38:15.069175 13134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:38:15.070161 12962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91: Bootstrap starting.
I20260810 06:38:15.070932 12962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:15.071370 12962 log.cc:824] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:15.072685 12962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91: No bootstrap required, opened a new log
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:38:15.074481 12962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.074680 12962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:15.074743 12962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a0f3e4cf084a1192db2966a0851b91, State: Initialized, Role: FOLLOWER
I20260810 06:38:15.074910 12962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.075340 12962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:38:15.077519 13152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:38:15.077644 13152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:38:15.077752 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.069541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44996 (local address 127.30.71.60:36929)
0810 06:38:15.069573 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.069575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.069583 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:38:15.069603 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.069603 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.069614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.069665 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.070010 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.070672 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.071647 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.071886 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.071928 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.074590 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.074591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.074591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.074599 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.076418 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.076854 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.076855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.076856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.076914 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.077111 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.077112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.077112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.077288 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.077338 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.077545 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.077631 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 06:38:15.078061 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.069459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:44996)
0810 06:38:15.069498 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:15.069510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:15.069549 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:15.069711 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:15.069712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:15.069872 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:15.069926 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.069928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.070770 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.070771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.071541 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.071545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.071988 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.071989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.072033 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.072521 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:15.072526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:15.074494 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:15.076515 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.076516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.076517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.076769 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.076983 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.076984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.076984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.077046 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.077401 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:15.077401 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:15.077442 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 06:38:15.077868 (+ 426us) client_negotiation.cc:240] Negotiation successful
0810 06:38:15.077967 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 06:38:15.078280 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.069621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.30.71.61:39575)
0810 06:38:15.069657 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.069659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.069696 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:38:15.069763 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.069764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.069773 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.069815 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.070379 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.070953 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.071884 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.072166 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.072246 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.074644 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.074645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.074646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.074653 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.076573 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.076983 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.076984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.076985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.077041 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.077238 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.077239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.077239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.077385 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.077603 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.078063 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.078180 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:38:15.079586 13134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: Bootstrap starting.
I20260810 06:38:15.080677 13134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:15.078368 12881 master_runner.cc:186] Error getting master registration for 127.30.71.61:39575: OK, instance_id { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" instance_seqno: 1786343894870489 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:38:15.080996 12881 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:38:15.081012 13134 log.cc:824] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:15.075296 13141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.081317 13141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:15.081979 13134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: No bootstrap required, opened a new log
I20260810 06:38:15.082329 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:38:15.084203 13134 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.084466 13134 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:15.084558 13134 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf4aedfe97b4476bbfa13a6eac21cdad, State: Initialized, Role: FOLLOWER
I20260810 06:38:15.084815 13134 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.085235 13134 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:38:15.087291 13174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:38:15.087394 13174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:38:15.087631 13162 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.087754 13162 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:15.088397 13056 master_runner.cc:186] Error getting master registration for 127.30.71.61:39575: OK, instance_id { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" instance_seqno: 1786343894870489 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:38:15.091832 13056 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:38:15.093811 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.084089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.30.71.61:39575)
0810 06:38:15.084136 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.084138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.084147 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:38:15.084232 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.084234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.084245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.084288 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.085207 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.085977 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.088597 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.088895 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.088938 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.090606 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.090607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.090608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.090616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.092290 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.092741 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.092743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.092743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.092830 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.093098 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.093099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.093100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.093263 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.093357 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.093601 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.093680 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:38:15.096719 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.084404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.30.71.62:46203)
0810 06:38:15.084446 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.084448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.084925 (+ 477us) server_negotiation.cc:407] Connection header received
0810 06:38:15.085026 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.085027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.085039 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.085071 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.085779 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.086479 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.087535 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.087811 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.087854 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.093279 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.093282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.093282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.093290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.095469 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.095858 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.095859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.095859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.095917 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.096161 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.096162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.096163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.096280 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.096382 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.096513 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.096588 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:38:15.100119 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.084722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.30.71.60:36929)
0810 06:38:15.084762 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.084764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.088008 (+ 3244us) server_negotiation.cc:407] Connection header received
0810 06:38:15.088108 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.088109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.088118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.088477 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.089526 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.090251 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.091776 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.092082 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.092123 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.094028 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.094029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.094029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.094037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.095884 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.099153 (+ 3269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.099155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.099155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.099279 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.099513 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.099514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.099514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.099640 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.099776 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.099916 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.099999 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:38:15.105631 13049 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } attempt: 3
Aug 10 06:38:15 dist-test-slave-w206 krb5kdc[12867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343894, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
I20260810 06:38:15.113229 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.106002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:36929 (local address 127.0.0.1:45016)
0810 06:38:15.106060 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:15.106071 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:15.106109 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:15.106283 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:15.106284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:15.106486 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:15.106538 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.106541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.107326 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.107327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.107842 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:15.107844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.108244 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.108245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:15.108302 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.108797 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:15.108803 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:15.110184 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:15.112054 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.112056 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.112057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.112289 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.112512 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:15.112513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:15.112514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:15.112575 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:15.112887 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:15.112888 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:15.112939 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:38:15.113007 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:38:15.113129 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:38:15.113569 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:15.106063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.30.71.60:36929)
0810 06:38:15.106108 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:15.106110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:15.106121 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:38:15.106181 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:15.106183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:15.106195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:15.106241 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:15.106603 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.107237 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.107958 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:15.108155 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:15.108242 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:15.110282 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:15.110285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:15.110285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:15.110293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:15.111947 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.112360 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.112361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.112361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.112439 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:15.112647 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:15.112648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:15.112649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:15.112794 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:15.112858 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:15.113407 (+ 549us) server_negotiation.cc:299] Negotiation successful
0810 06:38:15.113486 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:38:15.115163 13049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: Bootstrap starting.
I20260810 06:38:15.115851 13049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:15.116211 13049 log.cc:824] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:15.117074 13049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: No bootstrap required, opened a new log
I20260810 06:38:15.119776 13049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.119956 13049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:15.120002 13049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99c53355b5a34b2f9afbcea33cac44ae, State: Initialized, Role: FOLLOWER
I20260810 06:38:15.120131 13049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.120461 13177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.120543 13049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:15.120613 13177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:38:15.122563 13191 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:38:15.122704 13191 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:38:15.124823 12970 master_runner.cc:263] No leader master found from master 99c53355b5a34b2f9afbcea33cac44ae
I20260810 06:38:15.124882 12970 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 06:38:15.211040 12903 tablet.cc:2369] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:15.258810 13141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:15.258970 13141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.259303 13141 leader_election.cc:288] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99c53355b5a34b2f9afbcea33cac44ae (127.30.71.61:39575), bf4aedfe97b4476bbfa13a6eac21cdad (127.30.71.60:36929)
I20260810 06:38:15.259625 13104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21a0f3e4cf084a1192db2966a0851b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" is_pre_election: true
I20260810 06:38:15.259683 13018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21a0f3e4cf084a1192db2966a0851b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99c53355b5a34b2f9afbcea33cac44ae" is_pre_election: true
I20260810 06:38:15.259816 13018 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a0f3e4cf084a1192db2966a0851b91 in term 0.
I20260810 06:38:15.259858 13104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a0f3e4cf084a1192db2966a0851b91 in term 0.
I20260810 06:38:15.260078 12900 leader_election.cc:302] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a0f3e4cf084a1192db2966a0851b91, bf4aedfe97b4476bbfa13a6eac21cdad; no voters:
I20260810 06:38:15.260251 13141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:15.260300 13141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:15.260322 13141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:15.261405 13141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.261545 13141 leader_election.cc:288] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [CANDIDATE]: Term 1 election: Requested vote from peers 99c53355b5a34b2f9afbcea33cac44ae (127.30.71.61:39575), bf4aedfe97b4476bbfa13a6eac21cdad (127.30.71.60:36929)
I20260810 06:38:15.261718 13104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21a0f3e4cf084a1192db2966a0851b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad"
I20260810 06:38:15.261813 13104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:15.261864 13018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21a0f3e4cf084a1192db2966a0851b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99c53355b5a34b2f9afbcea33cac44ae"
I20260810 06:38:15.261932 13018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:15.262749 13104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a0f3e4cf084a1192db2966a0851b91 in term 1.
I20260810 06:38:15.262753 13018 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a0f3e4cf084a1192db2966a0851b91 in term 1.
I20260810 06:38:15.262924 12900 leader_election.cc:302] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a0f3e4cf084a1192db2966a0851b91, bf4aedfe97b4476bbfa13a6eac21cdad; no voters:
I20260810 06:38:15.263036 13141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:15.263170 13141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [term 1 LEADER]: Becoming Leader. State: Replica: 21a0f3e4cf084a1192db2966a0851b91, State: Running, Role: LEADER
I20260810 06:38:15.263266 13141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } }
I20260810 06:38:15.263707 13195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a0f3e4cf084a1192db2966a0851b91. Latest consensus state: current_term: 1 leader_uuid: "21a0f3e4cf084a1192db2966a0851b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.263819 13195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:15.264010 13197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:15.264249 13197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:15.265682 13018 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [term 1 FOLLOWER]: Refusing update from remote peer 21a0f3e4cf084a1192db2966a0851b91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:15.265682 13104 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [term 1 FOLLOWER]: Refusing update from remote peer 21a0f3e4cf084a1192db2966a0851b91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:15.265978 13195 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:15.266068 13141 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:15.267374 13177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a0f3e4cf084a1192db2966a0851b91. Latest consensus state: current_term: 1 leader_uuid: "21a0f3e4cf084a1192db2966a0851b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.267468 13177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:15.267530 13162 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a0f3e4cf084a1192db2966a0851b91. Latest consensus state: current_term: 1 leader_uuid: "21a0f3e4cf084a1192db2966a0851b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.267616 13162 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:15.268240 13195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21a0f3e4cf084a1192db2966a0851b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.268340 13195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:15.268384 13198 mvcc.cc:205] Tried to move back new op lower bound from 7316864595006181376 to 7316864594998988800. Current Snapshot: MvccSnapshot[applied={T|T < 7316864595006181376}]
I20260810 06:38:15.268779 13200 mvcc.cc:205] Tried to move back new op lower bound from 7316864595006181376 to 7316864594998988800. Current Snapshot: MvccSnapshot[applied={T|T < 7316864595006181376}]
I20260810 06:38:15.268865 13177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21a0f3e4cf084a1192db2966a0851b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.268937 13177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:15.269357 13141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21a0f3e4cf084a1192db2966a0851b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.269454 13141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:15.269857 13162 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21a0f3e4cf084a1192db2966a0851b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a0f3e4cf084a1192db2966a0851b91" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46203 } } peers { permanent_uuid: "99c53355b5a34b2f9afbcea33cac44ae" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 39575 } } peers { permanent_uuid: "bf4aedfe97b4476bbfa13a6eac21cdad" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 36929 } } }
I20260810 06:38:15.269941 13199 mvcc.cc:205] Tried to move back new op lower bound from 7316864595006181376 to 7316864594998988800. Current Snapshot: MvccSnapshot[applied={T|T < 7316864595006181376}]
I20260810 06:38:15.269945 13162 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:38:15.270533 13197 catalog_manager.cc:1347] Generated new cluster ID: c36275206c494dae990cd29f517ef7ce
I20260810 06:38:15.270584 13197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:15.279588 13197 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:15.280202 13197 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:15.287145 13197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a0f3e4cf084a1192db2966a0851b91: Generated new TSK 0
I20260810 06:38:15.287392 13197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:38:16.087880 13174 catalog_manager.cc:1259] Loaded cluster ID: c36275206c494dae990cd29f517ef7ce
I20260810 06:38:16.087991 13174 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: loading cluster ID for follower catalog manager: success
I20260810 06:38:16.088833 13174 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: acquiring CA information for follower catalog manager: success
I20260810 06:38:16.089145 13174 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf4aedfe97b4476bbfa13a6eac21cdad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:38:16.123335 13191 catalog_manager.cc:1259] Loaded cluster ID: c36275206c494dae990cd29f517ef7ce
I20260810 06:38:16.123421 13191 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: loading cluster ID for follower catalog manager: success
I20260810 06:38:16.124318 13191 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: acquiring CA information for follower catalog manager: success
I20260810 06:38:16.124655 13191 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 99c53355b5a34b2f9afbcea33cac44ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:38:16.350310 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 12881
I20260810 06:38:16.358074 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 12970
I20260810 06:38:16.365111 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 13056
2026-08-10T06:38:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3045 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:16 dist-test-slave-w206 krb5kdc[13212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:16 dist-test-slave-w206 krb5kdc[13212](info): set up 2 sockets
Aug 10 06:38:16 dist-test-slave-w206 krb5kdc[13212](info): commencing operation
krb5kdc: starting...
W20260810 06:38:17.492863 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:38:17 dist-test-slave-w206 krb5kdc[13212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:38:17.626780 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:40833
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:17.711812 13225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:17.712087 13225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:17.712131 13225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:17.713384 13225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:17.713469 13225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:17.713495 13225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:17.713524 13225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:17.713555 13225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:17.715167 13225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:40833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:17.715466 13225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:17.715708 13225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:17.720999 13230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:17.721011 13233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:17.721004 13231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:17.721150 13225 server_base.cc:1034] running on GCE node
I20260810 06:38:17.721622 13225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:17.721992 13225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:17.723166 13225 hybrid_clock.cc:648] HybridClock initialized: now 1786343897723137 us; error 45 us; skew 500 ppm
Aug 10 06:38:17 dist-test-slave-w206 krb5kdc[13212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343897, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:17.726063 13225 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:17.726528 13225 webserver.cc:460] Webserver started at http://127.30.71.62:45889/ using document root <none> and password file <none>
I20260810 06:38:17.726768 13225 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:17.726836 13225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:17.727059 13225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:17.728185 13225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "cb5605a5a85e4e339d6e3dc3a20191d8"
format_stamp: "Formatted at 2026-08-10 06:38:17 on dist-test-slave-w206"
server_key: "25616ec34d3551a580109bc86ac4b0e8"
server_key_iv: "7849ba295a97912818a6b6a18a3c0961"
server_key_version: "encryptionkey@0"
I20260810 06:38:17.728574 13225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "cb5605a5a85e4e339d6e3dc3a20191d8"
format_stamp: "Formatted at 2026-08-10 06:38:17 on dist-test-slave-w206"
server_key: "25616ec34d3551a580109bc86ac4b0e8"
server_key_iv: "7849ba295a97912818a6b6a18a3c0961"
server_key_version: "encryptionkey@0"
I20260810 06:38:17.730007 13225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:38:17.730794 13241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:17.731051 13225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:17.731168 13225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "cb5605a5a85e4e339d6e3dc3a20191d8"
format_stamp: "Formatted at 2026-08-10 06:38:17 on dist-test-slave-w206"
server_key: "25616ec34d3551a580109bc86ac4b0e8"
server_key_iv: "7849ba295a97912818a6b6a18a3c0961"
server_key_version: "encryptionkey@0"
I20260810 06:38:17.731261 13225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:17.751219 13225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:17.753603 13225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:17.753789 13225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:17.758062 13225 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:40833
I20260810 06:38:17.758100 13299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:40833 every 8 connection(s)
I20260810 06:38:17.758487 13225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:17.759472 13300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:17.761797 13300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8: Bootstrap starting.
I20260810 06:38:17.762354 13300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:17.762599 13300 log.cc:824] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:17.762678 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 13225
I20260810 06:38:17.762786 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:38:17.763027 31004 external_mini_cluster.cc:1428] Setting key 0f4b44e9671f7b8faa3ab1e240ee9ac2
I20260810 06:38:17.763521 13300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8: No bootstrap required, opened a new log
I20260810 06:38:17.765959 13300 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40833 } }
I20260810 06:38:17.766132 13300 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:17.766184 13300 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb5605a5a85e4e339d6e3dc3a20191d8, State: Initialized, Role: FOLLOWER
I20260810 06:38:17.766362 13300 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40833 } }
I20260810 06:38:17.766461 13300 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:17.766507 13300 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:17.766557 13300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:17.767253 13300 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40833 } }
I20260810 06:38:17.767377 13300 leader_election.cc:302] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb5605a5a85e4e339d6e3dc3a20191d8; no voters:
I20260810 06:38:17.767580 13300 leader_election.cc:288] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:17.767637 13305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:17.767791 13305 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [term 1 LEADER]: Becoming Leader. State: Replica: cb5605a5a85e4e339d6e3dc3a20191d8, State: Running, Role: LEADER
I20260810 06:38:17.767863 13300 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:17.767997 13305 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40833 } }
I20260810 06:38:17.768409 13305 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb5605a5a85e4e339d6e3dc3a20191d8. Latest consensus state: current_term: 1 leader_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40833 } } }
I20260810 06:38:17.768489 13305 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:17.768570 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5605a5a85e4e339d6e3dc3a20191d8" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40833 } } }
I20260810 06:38:17.768654 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:17.768862 13315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:17.769394 13315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:38:17 dist-test-slave-w206 krb5kdc[13212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:17.771704 13323 mvcc.cc:205] Tried to move back new op lower bound from 7316864605267456000 to 7316864605258301440. Current Snapshot: MvccSnapshot[applied={T|T < 7316864605267456000}]
I20260810 06:38:17.772601 13315 catalog_manager.cc:1347] Generated new cluster ID: 864877575076480c95a0f68ffb084d6d
I20260810 06:38:17.772665 13315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:17.775923 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:17.763589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50638 (local address 127.30.71.62:40833)
0810 06:38:17.763758 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:17.763769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:17.763800 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:38:17.764843 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:17.764866 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:17.764923 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:17.765047 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:17.765907 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.767282 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.768428 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:17.769069 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:17.769160 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:17.772306 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:17.772309 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:17.772309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:17.772320 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:17.774200 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:17.774637 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:17.774640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:17.774640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:17.774733 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:17.775002 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:17.775003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:17.775004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:17.775263 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:17.775435 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:17.775581 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:38:17.775732 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260810 06:38:17.777024 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:38:17.778106 13315 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:17.778620 13315 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:17.783591 13315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb5605a5a85e4e339d6e3dc3a20191d8: Generated new TSK 0
I20260810 06:38:17.783789 13315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:38:17.831363 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 13225
2026-08-10T06:38:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1464 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:17 dist-test-slave-w206 krb5kdc[13330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:17 dist-test-slave-w206 krb5kdc[13330](info): set up 2 sockets
Aug 10 06:38:17 dist-test-slave-w206 krb5kdc[13330](info): commencing operation
krb5kdc: starting...
W20260810 06:38:18.911019 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:38:19.046293 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33089
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:33089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:19.131004 13343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:19.131285 13343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:19.131331 13343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:19.132633 13343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:19.132725 13343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:19.132750 13343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:19.132769 13343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:19.132824 13343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:19.134460 13343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:33089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:19.134778 13343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:19.135066 13343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:19.140461 13350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:19.140512 13343 server_base.cc:1034] running on GCE node
W20260810 06:38:19.140486 13349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:19.140491 13352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:19.140962 13343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:19.141299 13343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:19.142513 13343 hybrid_clock.cc:648] HybridClock initialized: now 1786343899142457 us; error 112 us; skew 500 ppm
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:19.145336 13343 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:19.145917 13343 webserver.cc:460] Webserver started at http://127.30.71.62:40153/ using document root <none> and password file <none>
I20260810 06:38:19.146167 13343 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:19.146241 13343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:19.146404 13343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:19.147640 13343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "b871456921a547b4b807181e1084939b"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "d0a774b4801eb2179a743357ae625532"
server_key_iv: "c7d7301b95fa032af7dd1ea95d3cfa2a"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.148027 13343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "b871456921a547b4b807181e1084939b"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "d0a774b4801eb2179a743357ae625532"
server_key_iv: "c7d7301b95fa032af7dd1ea95d3cfa2a"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.149508 13343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:38:19.150343 13359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.150599 13343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:19.150724 13343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "b871456921a547b4b807181e1084939b"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "d0a774b4801eb2179a743357ae625532"
server_key_iv: "c7d7301b95fa032af7dd1ea95d3cfa2a"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.150810 13343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:19.166072 13343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:19.168870 13343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:19.169080 13343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:19.174124 13343 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:33089
I20260810 06:38:19.174161 13414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:33089 every 8 connection(s)
I20260810 06:38:19.174609 13343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:19.175931 13415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.178675 13415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Bootstrap starting.
I20260810 06:38:19.179436 13415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.179765 13415 log.cc:824] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:19.180734 13415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: No bootstrap required, opened a new log
I20260810 06:38:19.182353 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 13343
I20260810 06:38:19.182478 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:38:19.182672 31004 external_mini_cluster.cc:1428] Setting key fa8d5e9eaa34983db05e197d84487f18
I20260810 06:38:19.182825 13415 raft_consensus.cc:348] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:19.183064 13415 raft_consensus.cc:374] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.183095 13415 raft_consensus.cc:729] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b871456921a547b4b807181e1084939b, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.183267 13415 consensus_queue.cc:260] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:19.183344 13415 raft_consensus.cc:388] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:19.183403 13415 raft_consensus.cc:482] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:19.183467 13415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.184587 13415 raft_consensus.cc:504] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:19.184724 13415 leader_election.cc:302] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b871456921a547b4b807181e1084939b; no voters:
I20260810 06:38:19.184928 13415 leader_election.cc:288] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:19.185026 13421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:19.185190 13421 raft_consensus.cc:686] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 LEADER]: Becoming Leader. State: Replica: b871456921a547b4b807181e1084939b, State: Running, Role: LEADER
I20260810 06:38:19.185322 13415 sys_catalog.cc:564] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:19.185376 13421 consensus_queue.cc:237] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:19.185860 13422 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b871456921a547b4b807181e1084939b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } } }
I20260810 06:38:19.185940 13422 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:19.186138 13423 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b871456921a547b4b807181e1084939b. Latest consensus state: current_term: 1 leader_uuid: "b871456921a547b4b807181e1084939b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } } }
I20260810 06:38:19.186233 13423 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:19.186496 13430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:19.187117 13430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:19.188882 13430 catalog_manager.cc:1347] Generated new cluster ID: a34de976042c49ebb26ccad9ad69e9e6
I20260810 06:38:19.188941 13430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:19.194487 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.183563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.30.71.62:33089)
0810 06:38:19.183774 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.183783 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.183825 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:38:19.183948 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.183954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.183999 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.184119 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:19.185051 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.186594 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.187735 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.188353 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.188453 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.191105 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:19.191109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:19.191109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:19.191120 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:19.192925 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.193325 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.193328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.193328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.193433 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.193645 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.193646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.193646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.193901 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:19.194084 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.194151 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.194312 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260810 06:38:19.198017 13430 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:19.198720 13430 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:19.212277 13430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Generated new TSK 0
I20260810 06:38:19.212564 13430 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:38:19.246373 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33089
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:19.354193 13444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:19.354454 13444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:19.354501 13444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:19.356241 13444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:19.356353 13444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:19.356416 13444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:38:19.358467 13444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:19.358807 13444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:19.359155 13444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:19.364816 13452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:19.365044 13444 server_base.cc:1034] running on GCE node
W20260810 06:38:19.364823 13449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:19.364812 13450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:19.365401 13444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:19.365629 13444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:19.366775 13444 hybrid_clock.cc:648] HybridClock initialized: now 1786343899366760 us; error 28 us; skew 500 ppm
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:19.369539 13444 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:38:19.370038 13444 webserver.cc:460] Webserver started at http://127.30.71.1:43623/ using document root <none> and password file <none>
I20260810 06:38:19.370303 13444 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:19.370381 13444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:19.370537 13444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:19.371702 13444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "b8843ac6f70a4c2e804f69c52ea23203"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "270736b053570348cc87f60d17659f4e"
server_key_iv: "2bce2e9aa2ff8ddff45ffe8450cb3476"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.372078 13444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "b8843ac6f70a4c2e804f69c52ea23203"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "270736b053570348cc87f60d17659f4e"
server_key_iv: "2bce2e9aa2ff8ddff45ffe8450cb3476"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.373490 13444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:38:19.374289 13463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.374475 13444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:19.374586 13444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "b8843ac6f70a4c2e804f69c52ea23203"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "270736b053570348cc87f60d17659f4e"
server_key_iv: "2bce2e9aa2ff8ddff45ffe8450cb3476"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.374665 13444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:19.403465 13444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:19.405539 13444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:19.405725 13444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:19.406004 13444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:19.406347 13444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:19.406389 13444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.406446 13444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:19.406481 13444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.412410 13444 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38945
I20260810 06:38:19.412499 13593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38945 every 8 connection(s)
I20260810 06:38:19.412863 13444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:19.413758 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 13444
I20260810 06:38:19.413880 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:19.414093 31004 external_mini_cluster.cc:1428] Setting key 0d2d1c9a797d2962e6addc273d4fb564
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:38:19.423622 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.413863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:42425 (local address 127.30.71.62:33089)
0810 06:38:19.413911 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.413914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.414200 (+ 286us) server_negotiation.cc:407] Connection header received
0810 06:38:19.414889 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.414890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.414905 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.414950 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:19.415976 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.416483 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.417503 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.417713 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.417800 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.420475 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:19.420477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:19.420477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:19.420485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:19.422023 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.422513 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.422515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.422515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.422580 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.422852 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.422853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.422853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.423036 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:19.423124 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.423430 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.423525 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:38:19.423823 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.413868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:42425)
0810 06:38:19.414065 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.414104 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.414784 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.415045 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.415049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.415660 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:19.415861 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.415865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.416580 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.416581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.417405 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.417407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.417814 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.417815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.417956 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.418582 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:19.418590 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:19.420341 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:19.422147 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.422156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.422159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.422417 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.422652 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.422653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.422653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.422773 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.423114 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:19.423122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:19.423276 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.423405 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.423579 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260810 06:38:19.424541 13594 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33089
I20260810 06:38:19.424649 13594 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:19.424815 13594 heartbeater.cc:511] Master 127.30.71.62:33089 requested a full tablet report, sending...
I20260810 06:38:19.425382 13378 ts_manager.cc:194] Registered new tserver with Master: b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
I20260810 06:38:19.426101 13378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:42425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:38:19.442940 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33089
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:19.534242 13600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:19.534507 13600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:19.534535 13600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:19.535959 13600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:19.536021 13600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:19.536053 13600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:38:19.537830 13600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:19.538126 13600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:19.538414 13600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:19.544082 13605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:19.544111 13606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:19.544088 13608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:19.544458 13600 server_base.cc:1034] running on GCE node
I20260810 06:38:19.544714 13600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:19.545017 13600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:19.546291 13600 hybrid_clock.cc:648] HybridClock initialized: now 1786343899546273 us; error 53 us; skew 500 ppm
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:19.549381 13600 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:38:19.549942 13600 webserver.cc:460] Webserver started at http://127.30.71.2:36757/ using document root <none> and password file <none>
I20260810 06:38:19.550235 13600 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:19.550308 13600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:19.550465 13600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:19.551643 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "b04558c3273b41b9946fc0ffdadf7632"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "5012b409b7d52b6166657721415c764c"
server_key_iv: "b0234f6868046bc8cc68b8c8a368655b"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.552035 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "b04558c3273b41b9946fc0ffdadf7632"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "5012b409b7d52b6166657721415c764c"
server_key_iv: "b0234f6868046bc8cc68b8c8a368655b"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.553525 13600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:38:19.554446 13617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.554666 13600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:19.554755 13600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "b04558c3273b41b9946fc0ffdadf7632"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "5012b409b7d52b6166657721415c764c"
server_key_iv: "b0234f6868046bc8cc68b8c8a368655b"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.554831 13600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:19.574489 13600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:19.577057 13600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:19.577186 13600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:19.577570 13600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:19.577970 13600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:19.578019 13600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.578044 13600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:19.578087 13600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.584227 13600 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:45115
I20260810 06:38:19.584276 13740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:45115 every 8 connection(s)
I20260810 06:38:19.584641 13600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:19.589212 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 13600
I20260810 06:38:19.589362 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:19.589555 31004 external_mini_cluster.cc:1428] Setting key 7a389e239dff014b4c4f5d0b6b765c66
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:19.595109 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.585483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:51765 (local address 127.30.71.62:33089)
0810 06:38:19.585585 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.585587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.585839 (+ 252us) server_negotiation.cc:407] Connection header received
0810 06:38:19.586538 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.586539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.586560 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.586623 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:19.587760 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.588297 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.589321 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.589592 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.589675 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.592034 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:19.592037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:19.592037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:19.592045 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:19.593530 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.593998 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.594000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.594000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.594066 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.594379 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.594380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.594380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.594511 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:19.594589 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.594890 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.595013 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:38:19.595322 13743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.585494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:51765)
0810 06:38:19.585672 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.585710 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.586274 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.586756 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.586760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.587451 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:19.587662 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.587668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.588384 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.588384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.589211 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.589213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.589684 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.589685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.589854 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.590451 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:19.590459 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:19.591922 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:19.593651 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.593659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.593662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.593915 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.594138 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.594139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.594139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.594264 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.594589 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:19.594598 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:19.594749 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.594868 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.595064 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260810 06:38:19.596019 13741 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33089
I20260810 06:38:19.596113 13741 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:19.596383 13741 heartbeater.cc:511] Master 127.30.71.62:33089 requested a full tablet report, sending...
I20260810 06:38:19.596843 13378 ts_manager.cc:194] Registered new tserver with Master: b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115)
I20260810 06:38:19.597450 13378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:51765
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:38:19.622206 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33089
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:19.716681 13746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:19.716917 13746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:19.716946 13746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:19.718225 13746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:19.718282 13746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:19.718339 13746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:38:19.720012 13746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:19.720305 13746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:19.720562 13746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:19.726095 13755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:19.726090 13753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:19.726095 13752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:19.726462 13746 server_base.cc:1034] running on GCE node
I20260810 06:38:19.726670 13746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:19.726890 13746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:19.728042 13746 hybrid_clock.cc:648] HybridClock initialized: now 1786343899728037 us; error 43 us; skew 500 ppm
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:19.731081 13746 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:38:19.731552 13746 webserver.cc:460] Webserver started at http://127.30.71.3:39709/ using document root <none> and password file <none>
I20260810 06:38:19.731756 13746 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:19.731802 13746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:19.731901 13746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:19.732985 13746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "9b9a856fe00a4e219718092b71fc2db2"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "572b69f81ec3631ea329b77ab1909599"
server_key_iv: "a55b883183e69587cb32abf8c8a0a3b3"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.733350 13746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "9b9a856fe00a4e219718092b71fc2db2"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "572b69f81ec3631ea329b77ab1909599"
server_key_iv: "a55b883183e69587cb32abf8c8a0a3b3"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.734823 13746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:38:19.735868 13764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.736074 13746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:19.736168 13746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "9b9a856fe00a4e219718092b71fc2db2"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "572b69f81ec3631ea329b77ab1909599"
server_key_iv: "a55b883183e69587cb32abf8c8a0a3b3"
server_key_version: "encryptionkey@0"
I20260810 06:38:19.736229 13746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:19.754806 13746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:19.757191 13746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:19.757316 13746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:19.757516 13746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:19.757905 13746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:19.757956 13746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.757982 13746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:19.757993 13746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:19.764523 13746 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:38107
I20260810 06:38:19.764604 13888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:38107 every 8 connection(s)
I20260810 06:38:19.764978 13746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:19.768577 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 13746
I20260810 06:38:19.768661 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:19.768832 31004 external_mini_cluster.cc:1428] Setting key 7d0143d234e9493489039d509bbabfb3
Aug 10 06:38:19 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:19.775108 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.765816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:54733 (local address 127.30.71.62:33089)
0810 06:38:19.765893 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.765895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.766168 (+ 273us) server_negotiation.cc:407] Connection header received
0810 06:38:19.766687 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.766688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.766700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.766752 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:19.767827 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.768323 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.769817 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.770010 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.770062 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.772391 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:19.772393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:19.772393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:19.772400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:19.773667 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.774067 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.774067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.774068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.774121 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.774415 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.774415 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.774416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.774538 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:19.774598 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.774905 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.775022 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:38:19.775420 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.765792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:54733)
0810 06:38:19.766023 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.766067 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.766595 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.766820 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.766823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.767529 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:19.767738 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.767743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.768485 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.768486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.769688 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.769691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.770085 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.770086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.770201 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.770790 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:19.770798 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:19.772284 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:19.773748 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.773755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.773756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.773997 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.774209 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:19.774210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:19.774210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:19.774343 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:19.774611 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:19.774617 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:19.774778 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.774879 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.775107 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":143,"threads_started":1}
I20260810 06:38:19.776010 13889 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33089
I20260810 06:38:19.776127 13889 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:19.776315 13889 heartbeater.cc:511] Master 127.30.71.62:33089 requested a full tablet report, sending...
I20260810 06:38:19.776803 13378 ts_manager.cc:194] Registered new tserver with Master: 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107)
I20260810 06:38:19.777240 13378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:54733
I20260810 06:38:19.780655 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:19.788936 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.782092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.30.71.62:33089)
0810 06:38:19.782170 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.782172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.782226 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:38:19.782287 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.782288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.782297 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.782345 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:19.782947 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.783500 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.784221 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.784400 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.784451 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.785887 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:19.785888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:19.785888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:19.785894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:19.787580 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.787974 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.787975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.787975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.788039 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:19.788266 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:19.788267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:19.788268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:19.788456 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:19.788545 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.788727 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.788804 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:38:19.790716 13378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:19.792552 13378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:19.799172 13904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.795173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.30.71.1:38945)
0810 06:38:19.795340 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.795346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.795360 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:38:19.795502 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.795505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.795521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.795611 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.795829 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.796731 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.797931 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.798665 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.798709 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.798777 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.799030 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.799076 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260810 06:38:19.799160 13903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.795076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38945 (local address 127.0.0.1:41236)
0810 06:38:19.795204 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.795247 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.795367 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.795616 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.795619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.795622 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.795704 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.795706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.796827 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.796827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.797812 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.797815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.798788 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.798789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.798884 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.798895 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.799025 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.799084 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":83,"threads_started":1}
I20260810 06:38:19.799778 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.795599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.30.71.3:38107)
0810 06:38:19.795778 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.795782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.795888 (+ 106us) server_negotiation.cc:407] Connection header received
0810 06:38:19.796003 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.796005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.796020 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.796073 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.796312 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.797406 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.798557 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.799321 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.799393 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.799445 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.799651 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.799692 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260810 06:38:19.799875 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.795652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.30.71.2:45115)
0810 06:38:19.795853 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.795857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.795868 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:38:19.795909 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.795911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.795927 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.796003 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.796200 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.797368 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.798751 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.799503 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.799574 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.799628 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.799737 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.799776 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
I20260810 06:38:19.800052 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.795492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38107 (local address 127.0.0.1:39728)
0810 06:38:19.795827 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.795837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.795914 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.796175 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.796175 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.796178 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.796225 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.796226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.797517 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.797518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.798450 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.798453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.799483 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.799484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.799552 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.799557 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.799928 (+ 371us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.799985 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":68,"threads_started":1}
I20260810 06:38:19.800105 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.795314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:45115 (local address 127.0.0.1:37440)
0810 06:38:19.795443 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.795453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.795509 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.796088 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.796089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.796091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.796138 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.796140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.797492 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.797493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.798643 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.798647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.799589 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.799590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.799659 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.799664 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.799727 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.799766 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":52,"thread_start_us":86,"threads_started":1}
I20260810 06:38:19.800274 13517 tablet_service.cc:1467] Processing CreateTablet for tablet a0fdf121400a4848bc25145c646f811a (DEFAULT_TABLE table=test-table [id=5e5561530c76404799fd663f4299c8ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:19.800729 13517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fdf121400a4848bc25145c646f811a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.800978 13817 tablet_service.cc:1467] Processing CreateTablet for tablet a0fdf121400a4848bc25145c646f811a (DEFAULT_TABLE table=test-table [id=5e5561530c76404799fd663f4299c8ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:19.801225 13817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fdf121400a4848bc25145c646f811a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.801133 13672 tablet_service.cc:1467] Processing CreateTablet for tablet a0fdf121400a4848bc25145c646f811a (DEFAULT_TABLE table=test-table [id=5e5561530c76404799fd663f4299c8ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:19.801451 13672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0fdf121400a4848bc25145c646f811a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:19.803920 13910 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Bootstrap starting.
I20260810 06:38:19.804636 13910 tablet_bootstrap.cc:654] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.804853 13912 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Bootstrap starting.
I20260810 06:38:19.805238 13910 log.cc:824] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:19.805487 13912 tablet_bootstrap.cc:654] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.805624 13911 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap starting.
I20260810 06:38:19.806008 13912 log.cc:824] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:19.806056 13910 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: No bootstrap required, opened a new log
I20260810 06:38:19.806116 13910 ts_tablet_manager.cc:1397] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:19.806277 13911 tablet_bootstrap.cc:654] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:19.806790 13912 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: No bootstrap required, opened a new log
I20260810 06:38:19.806926 13912 ts_tablet_manager.cc:1397] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:19.806945 13911 log.cc:824] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:19.807767 13911 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: No bootstrap required, opened a new log
I20260810 06:38:19.807848 13911 ts_tablet_manager.cc:1397] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:38:19.808348 13910 raft_consensus.cc:348] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.808560 13910 raft_consensus.cc:374] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.808624 13910 raft_consensus.cc:729] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9a856fe00a4e219718092b71fc2db2, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.808642 13912 raft_consensus.cc:348] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.808773 13910 consensus_queue.cc:260] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.808833 13912 raft_consensus.cc:374] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.808876 13912 raft_consensus.cc:729] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04558c3273b41b9946fc0ffdadf7632, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.809013 13912 consensus_queue.cc:260] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.809087 13910 ts_tablet_manager.cc:1428] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:19.809204 13889 heartbeater.cc:503] Master 127.30.71.62:33089 was elected leader, sending a full tablet report...
I20260810 06:38:19.809319 13741 heartbeater.cc:503] Master 127.30.71.62:33089 was elected leader, sending a full tablet report...
I20260810 06:38:19.809309 13912 ts_tablet_manager.cc:1428] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:19.809945 13911 raft_consensus.cc:348] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.810112 13911 raft_consensus.cc:374] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:19.810156 13911 raft_consensus.cc:729] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8843ac6f70a4c2e804f69c52ea23203, State: Initialized, Role: FOLLOWER
I20260810 06:38:19.810271 13911 consensus_queue.cc:260] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.810575 13911 ts_tablet_manager.cc:1428] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:38:19.810672 13594 heartbeater.cc:503] Master 127.30.71.62:33089 was elected leader, sending a full tablet report...
W20260810 06:38:19.835325 13742 tablet.cc:2369] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:19.913718 13595 tablet.cc:2369] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:19.965585 13916 raft_consensus.cc:482] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:19.965765 13916 raft_consensus.cc:504] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.966193 13916 leader_election.cc:288] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945), b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115)
I20260810 06:38:19.969190 13904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.966542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:33405 (local address 127.30.71.1:38945)
0810 06:38:19.966598 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.966600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.966613 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:38:19.966651 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.966653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.966665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.966718 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.966892 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.967428 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.968427 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.968781 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.968891 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.968903 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.969079 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.969118 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 06:38:19.969190 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.966419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38945 (local address 127.30.71.3:33405)
0810 06:38:19.966490 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.966505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.966598 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.966740 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.966741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.966743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.966814 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.966815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.967516 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.967517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.968271 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.968273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.968897 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.968898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.968990 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.968993 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.969070 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.969113 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:38:19.969354 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.966643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:32789 (local address 127.30.71.2:45115)
0810 06:38:19.966700 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.966702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.966818 (+ 116us) server_negotiation.cc:407] Connection header received
0810 06:38:19.966893 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.966894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.966912 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.966975 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:19.967181 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.967638 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.968596 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.968988 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.969077 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.969087 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.969247 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.969289 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:38:19.969362 13920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.966550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:45115 (local address 127.30.71.3:32789)
0810 06:38:19.966750 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:19.966759 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:19.966829 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:19.967020 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:19.967021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:19.967023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:19.967104 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.967105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.967739 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.967741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.968491 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:19.968493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.969078 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.969079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:19.969165 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.969170 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:19.969240 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:38:19.969279 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":108,"threads_started":1}
I20260810 06:38:19.969650 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdf121400a4848bc25145c646f811a" candidate_uuid: "9b9a856fe00a4e219718092b71fc2db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8843ac6f70a4c2e804f69c52ea23203" is_pre_election: true
I20260810 06:38:19.969794 13692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdf121400a4848bc25145c646f811a" candidate_uuid: "9b9a856fe00a4e219718092b71fc2db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04558c3273b41b9946fc0ffdadf7632" is_pre_election: true
I20260810 06:38:19.969848 13543 raft_consensus.cc:2454] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b9a856fe00a4e219718092b71fc2db2 in term 0.
I20260810 06:38:19.969977 13692 raft_consensus.cc:2454] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b9a856fe00a4e219718092b71fc2db2 in term 0.
I20260810 06:38:19.970144 13766 leader_election.cc:302] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b9a856fe00a4e219718092b71fc2db2, b8843ac6f70a4c2e804f69c52ea23203; no voters:
I20260810 06:38:19.970296 13916 raft_consensus.cc:2781] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:19.970368 13916 raft_consensus.cc:482] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:19.970387 13916 raft_consensus.cc:3037] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.971263 13916 raft_consensus.cc:504] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.971415 13916 leader_election.cc:288] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [CANDIDATE]: Term 1 election: Requested vote from peers b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945), b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115)
I20260810 06:38:19.971691 13692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdf121400a4848bc25145c646f811a" candidate_uuid: "9b9a856fe00a4e219718092b71fc2db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04558c3273b41b9946fc0ffdadf7632"
I20260810 06:38:19.971707 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0fdf121400a4848bc25145c646f811a" candidate_uuid: "9b9a856fe00a4e219718092b71fc2db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8843ac6f70a4c2e804f69c52ea23203"
I20260810 06:38:19.971791 13692 raft_consensus.cc:3037] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.971791 13543 raft_consensus.cc:3037] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:19.972766 13543 raft_consensus.cc:2454] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b9a856fe00a4e219718092b71fc2db2 in term 1.
I20260810 06:38:19.972765 13692 raft_consensus.cc:2454] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b9a856fe00a4e219718092b71fc2db2 in term 1.
I20260810 06:38:19.972958 13766 leader_election.cc:302] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b9a856fe00a4e219718092b71fc2db2, b04558c3273b41b9946fc0ffdadf7632; no voters:
I20260810 06:38:19.973068 13916 raft_consensus.cc:2781] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:19.973224 13916 raft_consensus.cc:686] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 1 LEADER]: Becoming Leader. State: Replica: 9b9a856fe00a4e219718092b71fc2db2, State: Running, Role: LEADER
I20260810 06:38:19.973333 13916 consensus_queue.cc:237] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:19.974279 13377 catalog_manager.cc:5495] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3). New cstate: current_term: 1 leader_uuid: "9b9a856fe00a4e219718092b71fc2db2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } health_report { overall_health: HEALTHY } } }
I20260810 06:38:19.981346 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:19.978784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.30.71.3:38107)
0810 06:38:19.978852 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:19.978855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:19.978867 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:38:19.978915 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:19.978916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:19.978945 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:19.979070 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:19.979222 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.979698 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.980499 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:19.980701 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:19.980779 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:19.980966 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:19.981098 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:19.981183 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:19.981203 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 06:38:19.981224 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:38:19.983291 13543 raft_consensus.cc:1261] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Refusing update from remote peer 9b9a856fe00a4e219718092b71fc2db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:19.983291 13692 raft_consensus.cc:1261] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 1 FOLLOWER]: Refusing update from remote peer 9b9a856fe00a4e219718092b71fc2db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:19.983812 13916 consensus_queue.cc:1035] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.984012 13921 consensus_queue.cc:1035] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:19.986801 13925 mvcc.cc:205] Tried to move back new op lower bound from 7316864614328512512 to 7316864614291619840. Current Snapshot: MvccSnapshot[applied={T|T < 7316864614328512512}]
I20260810 06:38:19.987061 13926 mvcc.cc:205] Tried to move back new op lower bound from 7316864614328512512 to 7316864614291619840. Current Snapshot: MvccSnapshot[applied={T|T < 7316864614328512512}]
I20260810 06:38:19.989466 13377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:19.989580 13377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:19.990373 13377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Sending DeleteTablet for 3 replicas of tablet a0fdf121400a4848bc25145c646f811a
I20260810 06:38:19.990746 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 13444
I20260810 06:38:19.990736 13517 tablet_service.cc:1514] Processing DeleteTablet for tablet a0fdf121400a4848bc25145c646f811a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:41236
I20260810 06:38:19.991804 13672 tablet_service.cc:1514] Processing DeleteTablet for tablet a0fdf121400a4848bc25145c646f811a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:37440
I20260810 06:38:19.992831 13817 tablet_service.cc:1514] Processing DeleteTablet for tablet a0fdf121400a4848bc25145c646f811a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:39728
I20260810 06:38:19.993414 13936 tablet_replica.cc:330] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: stopping tablet replica
I20260810 06:38:19.993604 13936 raft_consensus.cc:2229] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:19.993764 13936 raft_consensus.cc:2258] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.994340 13936 ts_tablet_manager.cc:1905] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.995121 13937 tablet_replica.cc:330] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: stopping tablet replica
I20260810 06:38:19.995303 13937 raft_consensus.cc:2229] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:19.995491 13937 raft_consensus.cc:2258] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:19.995985 13937 ts_tablet_manager.cc:1905] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:19.996501 13936 ts_tablet_manager.cc:1918] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:19.996645 13936 log.cc:1196] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/a0fdf121400a4848bc25145c646f811a
I20260810 06:38:19.997030 13936 ts_tablet_manager.cc:1939] T a0fdf121400a4848bc25145c646f811a P b04558c3273b41b9946fc0ffdadf7632: Deleting consensus metadata
I20260810 06:38:19.997634 13361 catalog_manager.cc:4841] TS b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115): tablet a0fdf121400a4848bc25145c646f811a (table test-table [id=5e5561530c76404799fd663f4299c8ec]) successfully deleted
I20260810 06:38:19.998203 13937 ts_tablet_manager.cc:1918] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:19.998301 13937 log.cc:1196] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/a0fdf121400a4848bc25145c646f811a
I20260810 06:38:19.998651 13937 ts_tablet_manager.cc:1939] T a0fdf121400a4848bc25145c646f811a P 9b9a856fe00a4e219718092b71fc2db2: Deleting consensus metadata
I20260810 06:38:19.999225 13361 catalog_manager.cc:4841] TS 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107): tablet a0fdf121400a4848bc25145c646f811a (table test-table [id=5e5561530c76404799fd663f4299c8ec]) successfully deleted
W20260810 06:38:19.999730 13361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:38945 (error 108)
I20260810 06:38:19.999993 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 13600
I20260810 06:38:20.006193 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.005972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38945 (local address 127.0.0.1:41246)
0810 06:38:20.006067 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.006144 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 06:38:20.006386 13361 catalog_manager.cc:4584] TS b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0fdf121400a4848bc25145c646f811a: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38945: connect: Connection refused (error 111)
I20260810 06:38:20.009977 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 13746
I20260810 06:38:20.017243 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 13343
I20260810 06:38:20.024399 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33089
--webserver_interface=127.30.71.62
--webserver_port=40153
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:33089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:20.111187 13940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:20.111425 13940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:20.111451 13940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:20.112713 13940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:20.112771 13940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:20.112787 13940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:20.112799 13940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:20.112812 13940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:20.114480 13940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:33089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:20.114804 13940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:20.115144 13940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:20.125332 13946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:20.125324 13947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:20.125316 13949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:20.125686 13940 server_base.cc:1034] running on GCE node
I20260810 06:38:20.125978 13940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:20.126334 13940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:20.127516 13940 hybrid_clock.cc:648] HybridClock initialized: now 1786343900127493 us; error 40 us; skew 500 ppm
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:20.130349 13940 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:20.130796 13940 webserver.cc:460] Webserver started at http://127.30.71.62:40153/ using document root <none> and password file <none>
I20260810 06:38:20.131095 13940 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:20.131167 13940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:20.132608 13940 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:20.133288 13957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.133507 13940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:20.133610 13940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "b871456921a547b4b807181e1084939b"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "d0a774b4801eb2179a743357ae625532"
server_key_iv: "c7d7301b95fa032af7dd1ea95d3cfa2a"
server_key_version: "encryptionkey@0"
I20260810 06:38:20.134021 13940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:20.161934 13940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:20.164683 13940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:20.164830 13940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:20.169010 13940 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:33089
I20260810 06:38:20.169068 14013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:33089 every 8 connection(s)
I20260810 06:38:20.169499 13940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:20.170588 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 13940
I20260810 06:38:20.170915 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:38945
--local_ip_for_outbound_sockets=127.30.71.1
--tserver_master_addrs=127.30.71.62:33089
--webserver_port=43623
--webserver_interface=127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:20.171594 14014 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:38:20.172426 14014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Bootstrap starting.
I20260810 06:38:20.181962 14014 log.cc:824] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:20.184284 14014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:20.184640 14014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Bootstrap complete.
I20260810 06:38:20.187291 14014 raft_consensus.cc:348] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:20.187587 14014 raft_consensus.cc:729] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b871456921a547b4b807181e1084939b, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.187789 14014 consensus_queue.cc:260] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:20.187877 14014 raft_consensus.cc:388] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:20.187942 14014 raft_consensus.cc:482] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:20.187994 14014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:38:20.189136 14014 raft_consensus.cc:504] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:20.189273 14014 leader_election.cc:302] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b871456921a547b4b807181e1084939b; no voters:
I20260810 06:38:20.189522 14014 leader_election.cc:288] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:38:20.189565 14018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:38:20.189697 14018 raft_consensus.cc:686] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 LEADER]: Becoming Leader. State: Replica: b871456921a547b4b807181e1084939b, State: Running, Role: LEADER
I20260810 06:38:20.189843 14014 sys_catalog.cc:564] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:20.189872 14018 consensus_queue.cc:237] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:20.190428 14020 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b871456921a547b4b807181e1084939b. Latest consensus state: current_term: 2 leader_uuid: "b871456921a547b4b807181e1084939b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } } }
I20260810 06:38:20.190529 14020 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:20.190784 14019 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b871456921a547b4b807181e1084939b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } } }
I20260810 06:38:20.190865 14019 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:20.191226 14025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:20.192083 14025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:20.192281 14025 catalog_manager.cc:1259] Loaded cluster ID: a34de976042c49ebb26ccad9ad69e9e6
I20260810 06:38:20.192512 14025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:20.192818 14025 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 06:38:20.197494 14025 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 06:38:20.201162 14025 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:20.201438 14025 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Loaded TSK: 0
I20260810 06:38:20.201678 14025 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:38:20.265566 14016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:20.265856 14016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:20.265904 14016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:20.267349 14016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:20.267421 14016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:20.267467 14016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:38:20.269042 14016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:38945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=43623
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:20.269325 14016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:20.269574 14016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:20.274945 14045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:20.275068 14016 server_base.cc:1034] running on GCE node
W20260810 06:38:20.275121 14048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:20.274957 14046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:20.275391 14016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:20.275667 14016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:20.276818 14016 hybrid_clock.cc:648] HybridClock initialized: now 1786343900276799 us; error 35 us; skew 500 ppm
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:20.279759 14016 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:38:20.280253 14016 webserver.cc:460] Webserver started at http://127.30.71.1:43623/ using document root <none> and password file <none>
I20260810 06:38:20.280539 14016 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:20.280637 14016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:20.282264 14016 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:20.283074 14055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.283252 14016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:20.283386 14016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "b8843ac6f70a4c2e804f69c52ea23203"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "270736b053570348cc87f60d17659f4e"
server_key_iv: "2bce2e9aa2ff8ddff45ffe8450cb3476"
server_key_version: "encryptionkey@0"
I20260810 06:38:20.283756 14016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:20.293882 14016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:20.296520 14016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:20.296711 14016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:20.296976 14016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:20.297492 14064 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:38:20.299710 14016 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:38:20.299805 14016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 06:38:20.299847 14016 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:38:20.300558 14016 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:38:20.300616 14016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 06:38:20.300714 14064 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap starting.
I20260810 06:38:20.308154 14016 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38945
I20260810 06:38:20.308230 14181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38945 every 8 connection(s)
I20260810 06:38:20.308635 14016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:20.313153 14064 log.cc:824] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:20.317514 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 14016
I20260810 06:38:20.317826 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:45115
--local_ip_for_outbound_sockets=127.30.71.2
--tserver_master_addrs=127.30.71.62:33089
--webserver_port=36757
--webserver_interface=127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:20.319326 14064 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:20.319744 14064 tablet_bootstrap.cc:492] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap complete.
I20260810 06:38:20.321058 14064 ts_tablet_manager.cc:1397] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.008s
I20260810 06:38:20.321983 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.309565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:42081)
0810 06:38:20.309806 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.309866 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.310341 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.311705 (+ 1364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.311709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.312056 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:20.312258 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.312263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.313482 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.313482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.314266 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.314268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.314954 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.314955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.315148 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.315748 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:20.315759 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:20.317666 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:20.320226 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.320234 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.320237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.320472 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.320759 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.320760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.320761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.320875 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.321330 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:20.321336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:20.321475 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.321588 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.321773 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":128,"threads_started":1}
I20260810 06:38:20.321980 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.309658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:42081 (local address 127.30.71.62:33089)
0810 06:38:20.311275 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.311283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.311319 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:38:20.311463 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.311469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.311512 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.311674 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.312399 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.313328 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.314378 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.314814 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.314955 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.317823 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.317828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.317828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.317841 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.320114 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.320577 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.320581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.320581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.320682 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.320948 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.320949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.320949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.321253 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.321358 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.321640 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.321773 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1566,"thread_start_us":97,"threads_started":1}
I20260810 06:38:20.322494 14064 raft_consensus.cc:348] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:20.323014 14064 raft_consensus.cc:729] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8843ac6f70a4c2e804f69c52ea23203, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.323232 14182 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33089
I20260810 06:38:20.323207 14064 consensus_queue.cc:260] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } } peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } }
I20260810 06:38:20.323374 14182 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:20.323566 14064 ts_tablet_manager.cc:1428] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20260810 06:38:20.323570 14182 heartbeater.cc:511] Master 127.30.71.62:33089 requested a full tablet report, sending...
I20260810 06:38:20.324366 13975 ts_manager.cc:194] Registered new tserver with Master: b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945)
I20260810 06:38:20.325520 13975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:42081
I20260810 06:38:20.326502 14182 heartbeater.cc:503] Master 127.30.71.62:33089 was elected leader, sending a full tablet report...
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:38:20.336359 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.325256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.30.71.1:38945)
0810 06:38:20.325489 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.325494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.325503 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:38:20.325537 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.325539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.325556 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.326220 (+ 664us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.326456 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.327525 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.328841 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.329396 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.331092 (+ 1696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.332561 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.332571 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.332571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.332579 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.334681 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.335290 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.335294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.335294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.335364 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.335613 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.335614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.335614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.335810 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.335907 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.336135 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.336232 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":71,"threads_started":1}
I20260810 06:38:20.336452 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.324988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38945 (local address 127.0.0.1:41260)
0810 06:38:20.325210 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.325265 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.325348 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.325659 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.325665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.326035 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:20.326124 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.326127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.327625 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.327626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.328727 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.328733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.329508 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.329509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.329629 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.330270 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:20.330277 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:20.332446 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:20.334794 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.334803 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.334806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.335182 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.335451 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.335452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.335453 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.335536 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.335899 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:20.335907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:20.335977 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.336084 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.336295 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":113,"threads_started":1}
I20260810 06:38:20.337193 14112 tablet_service.cc:1514] Processing DeleteTablet for tablet a0fdf121400a4848bc25145c646f811a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:19 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:41260
I20260810 06:38:20.337733 14192 tablet_replica.cc:330] stopping tablet replica
I20260810 06:38:20.337854 14192 raft_consensus.cc:2229] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:20.337934 14192 raft_consensus.cc:2258] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:20.338290 14192 ts_tablet_manager.cc:1905] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:20.340965 14192 ts_tablet_manager.cc:1918] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:20.341058 14192 log.cc:1196] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/a0fdf121400a4848bc25145c646f811a
I20260810 06:38:20.341358 14192 ts_tablet_manager.cc:1939] T a0fdf121400a4848bc25145c646f811a P b8843ac6f70a4c2e804f69c52ea23203: Deleting consensus metadata
I20260810 06:38:20.341972 13959 catalog_manager.cc:4841] TS b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945): tablet a0fdf121400a4848bc25145c646f811a (table test-table [id=5e5561530c76404799fd663f4299c8ec]) successfully deleted
W20260810 06:38:20.414508 14187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:20.414785 14187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:20.414825 14187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:20.416108 14187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:20.416188 14187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:20.416235 14187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:38:20.417852 14187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:45115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=36757
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:20.418156 14187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:20.418437 14187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:20.423923 14198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:20.423928 14197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:20.423923 14200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:20.424206 14187 server_base.cc:1034] running on GCE node
I20260810 06:38:20.424435 14187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:20.424654 14187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:20.425799 14187 hybrid_clock.cc:648] HybridClock initialized: now 1786343900425793 us; error 36 us; skew 500 ppm
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:20.428644 14187 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:38:20.429152 14187 webserver.cc:460] Webserver started at http://127.30.71.2:36757/ using document root <none> and password file <none>
I20260810 06:38:20.429427 14187 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:20.429522 14187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:20.431074 14187 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:38:20.431778 14210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.432000 14187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:20.432114 14187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "b04558c3273b41b9946fc0ffdadf7632"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "5012b409b7d52b6166657721415c764c"
server_key_iv: "b0234f6868046bc8cc68b8c8a368655b"
server_key_version: "encryptionkey@0"
I20260810 06:38:20.432507 14187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:20.447741 14187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:20.450343 14187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:20.450526 14187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:20.450804 14187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:20.451200 14187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:20.451236 14187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.451259 14187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:20.451304 14187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.457666 14187 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:45115
I20260810 06:38:20.457732 14334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:45115 every 8 connection(s)
I20260810 06:38:20.458101 14187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:20.461158 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 14187
I20260810 06:38:20.461477 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:38107
--local_ip_for_outbound_sockets=127.30.71.3
--tserver_master_addrs=127.30.71.62:33089
--webserver_port=39709
--webserver_interface=127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:20.469401 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.458865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:52713 (local address 127.30.71.62:33089)
0810 06:38:20.458919 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.458921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.459183 (+ 262us) server_negotiation.cc:407] Connection header received
0810 06:38:20.459746 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.459747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.459765 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.459833 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.460862 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.461671 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.462766 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.463115 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.463177 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.465865 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.465867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.465867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.465877 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.467761 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.468219 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.468220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.468221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.468327 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.468600 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.468601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.468602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.468779 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.468865 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.469143 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.469264 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:38:20.469421 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.458845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:52713)
0810 06:38:20.459043 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.459090 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.459623 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.459928 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.459937 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.460544 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:20.460751 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.460756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.461791 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.461792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.462663 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.462666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.463216 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.463218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.463467 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.464151 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:20.464163 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:20.465735 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:20.467864 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.467874 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.467876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.468136 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.468411 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.468411 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.468412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.468531 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.468855 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:20.468860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:20.469002 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.469091 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.469229 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260810 06:38:20.470145 14335 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33089
I20260810 06:38:20.470286 14335 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:20.470520 14335 heartbeater.cc:511] Master 127.30.71.62:33089 requested a full tablet report, sending...
I20260810 06:38:20.471045 13975 ts_manager.cc:194] Registered new tserver with Master: b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115)
I20260810 06:38:20.471626 13975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:52713
W20260810 06:38:20.550642 14338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:20.550871 14338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:20.550896 14338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:20.552086 14338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:20.552134 14338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:20.552165 14338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:38:20.553679 14338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:38107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=39709
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:20.553896 14338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:20.554085 14338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:20.559819 14347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:20.559805 14345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:20.559877 14338 server_base.cc:1034] running on GCE node
W20260810 06:38:20.559820 14344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:20.560215 14338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:20.560516 14338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:20.561666 14338 hybrid_clock.cc:648] HybridClock initialized: now 1786343900561649 us; error 43 us; skew 500 ppm
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:20.564507 14338 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:38:20.564981 14338 webserver.cc:460] Webserver started at http://127.30.71.3:39709/ using document root <none> and password file <none>
I20260810 06:38:20.565222 14338 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:20.565281 14338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:20.566771 14338 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:38:20.567515 14355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.567705 14338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:38:20.567816 14338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "9b9a856fe00a4e219718092b71fc2db2"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "572b69f81ec3631ea329b77ab1909599"
server_key_iv: "a55b883183e69587cb32abf8c8a0a3b3"
server_key_version: "encryptionkey@0"
I20260810 06:38:20.568205 14338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:20.595423 14338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:20.598073 14338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:20.598268 14338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:20.598567 14338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:20.598929 14338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:20.599020 14338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.599119 14338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:20.599154 14338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.605463 14338 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:38107
I20260810 06:38:20.605553 14480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:38107 every 8 connection(s)
I20260810 06:38:20.605929 14338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:20.608103 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 14338
Aug 10 06:38:20 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:20.615751 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.608568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.30.71.62:33089)
0810 06:38:20.608696 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.608697 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.608744 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:38:20.608818 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.608818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.608832 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.608904 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.609613 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.610279 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.611114 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.611337 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.611407 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.612798 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.612800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.612800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.612808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.614452 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.614833 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.614834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.614835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.614905 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.615163 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.615164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.615164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.615293 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.615483 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.615532 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.615625 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260810 06:38:20.617162 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.606812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:41853 (local address 127.30.71.62:33089)
0810 06:38:20.606865 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.606867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.607109 (+ 242us) server_negotiation.cc:407] Connection header received
0810 06:38:20.607614 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.607615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.607627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.607693 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:20.609055 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.609787 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.611011 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.611255 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.611315 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.614077 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:20.614078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:20.614078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:20.614085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:20.615574 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.616147 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.616148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.616148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.616209 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:20.616485 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:20.616486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:20.616486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:20.616600 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:20.616679 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.616980 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.617069 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:38:20.617410 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.606765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:41853)
0810 06:38:20.606943 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.607014 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.607525 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.607751 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.607754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.608636 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:20.608949 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.608956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.609886 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.609887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.610886 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.610889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.611341 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.611342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.611528 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.612197 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:20.612206 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:20.613964 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:20.615688 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.615696 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.615698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.616067 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.616278 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:20.616279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:20.616279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:20.616413 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:20.616671 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:20.616678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:20.616842 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.616958 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.617136 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260810 06:38:20.617888 14481 heartbeater.cc:347] Connected to a master server at 127.30.71.62:33089
I20260810 06:38:20.618000 14481 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:20.618157 14481 heartbeater.cc:511] Master 127.30.71.62:33089 requested a full tablet report, sending...
I20260810 06:38:20.618587 13975 ts_manager.cc:194] Registered new tserver with Master: 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107)
I20260810 06:38:20.618939 13975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:41853
I20260810 06:38:20.627589 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:20.630785 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.628058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.30.71.62:33089)
0810 06:38:20.628144 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.628146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.628325 (+ 179us) server_negotiation.cc:407] Connection header received
0810 06:38:20.628386 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.628387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.628429 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.628509 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:20.628678 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.629242 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.630005 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.630200 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.630267 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.630451 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.630570 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.630655 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.630675 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.630698 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:38:20.631239 13975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:20.633155 13975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:20.635833 14112 tablet_service.cc:1467] Processing CreateTablet for tablet 1c532d0fda28419990ee101796d8a45d (DEFAULT_TABLE table=test-table [id=b9e91e156aa548b088ed585f8b0f00cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.636067 14112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c532d0fda28419990ee101796d8a45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.637876 14064 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap starting.
I20260810 06:38:20.638253 14064 tablet_bootstrap.cc:654] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.639529 14064 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: No bootstrap required, opened a new log
I20260810 06:38:20.639596 14064 ts_tablet_manager.cc:1397] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:38:20.639760 14064 raft_consensus.cc:348] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.639866 14064 raft_consensus.cc:374] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.639938 14064 raft_consensus.cc:729] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8843ac6f70a4c2e804f69c52ea23203, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.640002 14064 consensus_queue.cc:260] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.640125 14064 ts_tablet_manager.cc:1428] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:20.641156 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.635436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38107 (local address 127.0.0.1:39754)
0810 06:38:20.635552 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.635561 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.635643 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.636062 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.636063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.636066 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:20.636122 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.636124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.637596 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.637597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.638729 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.638731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.640882 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.640883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.640943 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.640968 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.641024 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.641071 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260810 06:38:20.641425 14489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.635650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39754 (local address 127.30.71.3:38107)
0810 06:38:20.635856 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.635861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.635871 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:38:20.635942 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.635945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.635962 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.636047 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:20.636248 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.637481 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.638818 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.639536 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.639654 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.639728 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.641290 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.641339 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":132,"threads_started":1}
I20260810 06:38:20.642057 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.635307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:45115 (local address 127.0.0.1:37454)
0810 06:38:20.635385 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.635397 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.635440 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.638168 (+ 2728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.638169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.638171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:20.638208 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.638209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.639683 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.639684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.640780 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.640783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.641850 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.641851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.641916 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.641919 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.641968 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.642009 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 06:38:20.642107 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.635436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.30.71.2:45115)
0810 06:38:20.637975 (+ 2539us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.637980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.637993 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:38:20.638043 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.638047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.638064 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.638162 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:20.638269 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.639517 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.640903 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.641675 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.641848 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.641918 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.641983 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.642023 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2495,"thread_start_us":135,"threads_started":1}
I20260810 06:38:20.642273 14406 tablet_service.cc:1467] Processing CreateTablet for tablet 1c532d0fda28419990ee101796d8a45d (DEFAULT_TABLE table=test-table [id=b9e91e156aa548b088ed585f8b0f00cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.642683 14406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c532d0fda28419990ee101796d8a45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.642890 14265 tablet_service.cc:1467] Processing CreateTablet for tablet 1c532d0fda28419990ee101796d8a45d (DEFAULT_TABLE table=test-table [id=b9e91e156aa548b088ed585f8b0f00cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:20.643275 14265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c532d0fda28419990ee101796d8a45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:20.646793 14491 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Bootstrap starting.
I20260810 06:38:20.646939 14492 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Bootstrap starting.
I20260810 06:38:20.647684 14491 tablet_bootstrap.cc:654] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.647691 14492 tablet_bootstrap.cc:654] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:20.648181 14491 log.cc:824] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:20.648208 14492 log.cc:824] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:20.649053 14491 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: No bootstrap required, opened a new log
I20260810 06:38:20.649150 14491 ts_tablet_manager.cc:1397] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:20.650805 14491 raft_consensus.cc:348] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.651054 14491 raft_consensus.cc:374] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.651103 14491 raft_consensus.cc:729] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9a856fe00a4e219718092b71fc2db2, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.651131 14492 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: No bootstrap required, opened a new log
I20260810 06:38:20.651194 14492 ts_tablet_manager.cc:1397] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260810 06:38:20.651252 14491 consensus_queue.cc:260] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.651654 14481 heartbeater.cc:503] Master 127.30.71.62:33089 was elected leader, sending a full tablet report...
I20260810 06:38:20.651676 14491 ts_tablet_manager.cc:1428] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:38:20.653246 14492 raft_consensus.cc:348] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.653396 14492 raft_consensus.cc:374] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:20.653419 14492 raft_consensus.cc:729] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04558c3273b41b9946fc0ffdadf7632, State: Initialized, Role: FOLLOWER
I20260810 06:38:20.653542 14492 consensus_queue.cc:260] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.653867 14492 ts_tablet_manager.cc:1428] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 06:38:20.653882 14335 heartbeater.cc:503] Master 127.30.71.62:33089 was elected leader, sending a full tablet report...
W20260810 06:38:20.708818 14336 tablet.cc:2369] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:20.811607 14183 tablet.cc:2369] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:20.856577 14482 tablet.cc:2369] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:20.921494 14498 raft_consensus.cc:482] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:20.921631 14498 raft_consensus.cc:504] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.922094 14498 leader_election.cc:288] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107), b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945)
I20260810 06:38:20.925334 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.922322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38107 (local address 127.30.71.2:35469)
0810 06:38:20.922383 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.922398 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.922455 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.922617 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.922618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.922620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:20.922695 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.922697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.923523 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.923524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.924456 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.924459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.925062 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.925063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.925150 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.925155 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.925210 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.925252 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 06:38:20.925446 14489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.922416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:35469 (local address 127.30.71.3:38107)
0810 06:38:20.922462 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.922465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.922492 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:38:20.922521 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.922522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.922534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.922579 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:20.922785 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.923441 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.924543 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.924974 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.925031 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.925040 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.925327 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.925364 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:38:20.925591 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.922454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38945 (local address 127.30.71.2:38659)
0810 06:38:20.922628 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:20.922637 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:20.922682 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:20.922917 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:20.922918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:20.922920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:20.923017 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.923019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.923690 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.923691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.924562 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:20.924565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.925323 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.925324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:20.925402 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.925407 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:20.925463 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 06:38:20.925506 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260810 06:38:20.925812 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.922572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:38659 (local address 127.30.71.1:38945)
0810 06:38:20.922689 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.922691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.922700 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:38:20.922774 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.922775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.922810 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.922879 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:20.923086 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.923602 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.924655 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.925097 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.925170 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.925258 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.925698 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.925749 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260810 06:38:20.925784 14426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c532d0fda28419990ee101796d8a45d" candidate_uuid: "b04558c3273b41b9946fc0ffdadf7632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9a856fe00a4e219718092b71fc2db2" is_pre_election: true
I20260810 06:38:20.925976 14426 raft_consensus.cc:2454] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b04558c3273b41b9946fc0ffdadf7632 in term 0.
I20260810 06:38:20.926107 14132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c532d0fda28419990ee101796d8a45d" candidate_uuid: "b04558c3273b41b9946fc0ffdadf7632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8843ac6f70a4c2e804f69c52ea23203" is_pre_election: true
I20260810 06:38:20.926263 14132 raft_consensus.cc:2454] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b04558c3273b41b9946fc0ffdadf7632 in term 0.
I20260810 06:38:20.926259 14213 leader_election.cc:302] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b9a856fe00a4e219718092b71fc2db2, b04558c3273b41b9946fc0ffdadf7632; no voters:
I20260810 06:38:20.926396 14498 raft_consensus.cc:2781] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:20.926491 14498 raft_consensus.cc:482] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:20.926527 14498 raft_consensus.cc:3037] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.927372 14498 raft_consensus.cc:504] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.927523 14498 leader_election.cc:288] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [CANDIDATE]: Term 1 election: Requested vote from peers 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107), b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945)
I20260810 06:38:20.927700 14132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c532d0fda28419990ee101796d8a45d" candidate_uuid: "b04558c3273b41b9946fc0ffdadf7632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8843ac6f70a4c2e804f69c52ea23203"
I20260810 06:38:20.927718 14426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c532d0fda28419990ee101796d8a45d" candidate_uuid: "b04558c3273b41b9946fc0ffdadf7632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9a856fe00a4e219718092b71fc2db2"
I20260810 06:38:20.927784 14132 raft_consensus.cc:3037] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.927809 14426 raft_consensus.cc:3037] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:20.928514 14132 raft_consensus.cc:2454] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b04558c3273b41b9946fc0ffdadf7632 in term 1.
I20260810 06:38:20.928516 14426 raft_consensus.cc:2454] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b04558c3273b41b9946fc0ffdadf7632 in term 1.
I20260810 06:38:20.928706 14213 leader_election.cc:302] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b04558c3273b41b9946fc0ffdadf7632, b8843ac6f70a4c2e804f69c52ea23203; no voters:
I20260810 06:38:20.928820 14498 raft_consensus.cc:2781] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:20.928997 14498 raft_consensus.cc:686] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 1 LEADER]: Becoming Leader. State: Replica: b04558c3273b41b9946fc0ffdadf7632, State: Running, Role: LEADER
I20260810 06:38:20.929107 14498 consensus_queue.cc:237] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:20.930056 13975 catalog_manager.cc:5495] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 reported cstate change: term changed from 0 to 1, leader changed from <none> to b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "b04558c3273b41b9946fc0ffdadf7632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:20.997550 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:20.995229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.30.71.2:45115)
0810 06:38:20.995298 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:20.995300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:20.995308 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:38:20.995345 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:20.995346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:20.995371 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:20.995438 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:20.995611 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.996097 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.996825 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:20.997023 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:20.997080 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:20.997228 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:20.997330 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:20.997399 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:20.997456 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 06:38:20.997475 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:38:20.999604 14426 raft_consensus.cc:1261] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 1 FOLLOWER]: Refusing update from remote peer b04558c3273b41b9946fc0ffdadf7632: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:20.999604 14132 raft_consensus.cc:1261] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Refusing update from remote peer b04558c3273b41b9946fc0ffdadf7632: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:21.000025 14498 consensus_queue.cc:1035] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:21.000188 14501 consensus_queue.cc:1035] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:21.002300 14504 mvcc.cc:205] Tried to move back new op lower bound from 7316864618490941440 to 7316864618206408704. Current Snapshot: MvccSnapshot[applied={T|T < 7316864618490941440}]
I20260810 06:38:21.002995 14506 mvcc.cc:205] Tried to move back new op lower bound from 7316864618490941440 to 7316864618206408704. Current Snapshot: MvccSnapshot[applied={T|T < 7316864618490941440}]
I20260810 06:38:21.005708 13974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:21.005805 13974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:21.006639 13974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Sending DeleteTablet for 3 replicas of tablet 1c532d0fda28419990ee101796d8a45d
I20260810 06:38:21.006994 14265 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c532d0fda28419990ee101796d8a45d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:21 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:37454
I20260810 06:38:21.007021 14406 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c532d0fda28419990ee101796d8a45d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:21 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:39754
I20260810 06:38:21.007081 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 14016
I20260810 06:38:21.011353 14515 tablet_replica.cc:330] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: stopping tablet replica
I20260810 06:38:21.011513 14515 raft_consensus.cc:2229] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:21.011642 14515 raft_consensus.cc:2258] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:21.012352 14515 ts_tablet_manager.cc:1905] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:21.014312 14515 ts_tablet_manager.cc:1918] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:21.014402 14515 log.cc:1196] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/1c532d0fda28419990ee101796d8a45d
I20260810 06:38:21.014793 14515 ts_tablet_manager.cc:1939] T 1c532d0fda28419990ee101796d8a45d P 9b9a856fe00a4e219718092b71fc2db2: Deleting consensus metadata
I20260810 06:38:21.015456 13959 catalog_manager.cc:4841] TS 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107): tablet 1c532d0fda28419990ee101796d8a45d (table test-table [id=b9e91e156aa548b088ed585f8b0f00cc]) successfully deleted
I20260810 06:38:21.017355 14514 tablet_replica.cc:330] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: stopping tablet replica
I20260810 06:38:21.017503 14514 raft_consensus.cc:2229] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:21.017663 14514 raft_consensus.cc:2258] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:21.018194 14514 ts_tablet_manager.cc:1905] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:21.019887 14514 ts_tablet_manager.cc:1918] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:21.019991 14514 log.cc:1196] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/1c532d0fda28419990ee101796d8a45d
I20260810 06:38:21.020272 14514 ts_tablet_manager.cc:1939] T 1c532d0fda28419990ee101796d8a45d P b04558c3273b41b9946fc0ffdadf7632: Deleting consensus metadata
I20260810 06:38:21.020854 13959 catalog_manager.cc:4841] TS b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115): tablet 1c532d0fda28419990ee101796d8a45d (table test-table [id=b9e91e156aa548b088ed585f8b0f00cc]) successfully deleted
W20260810 06:38:21.021809 13959 connection.cc:535] client connection to 127.30.71.1:38945 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 06:38:21.021905 13959 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:38:21.022197 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 14187
I20260810 06:38:21.023248 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.022273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38945 (local address 127.0.0.1:41274)
0810 06:38:21.023167 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.023212 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":871}
W20260810 06:38:21.026594 13959 catalog_manager.cc:4584] TS b8843ac6f70a4c2e804f69c52ea23203 (127.30.71.1:38945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c532d0fda28419990ee101796d8a45d: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38945: connect: Connection refused (error 111)
I20260810 06:38:21.030726 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 14338
I20260810 06:38:21.036065 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 13940
I20260810 06:38:21.044852 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33089
--webserver_interface=127.30.71.62
--webserver_port=40153
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:33089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:21.132429 14517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:21.132664 14517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:21.132689 14517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:21.133849 14517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:21.133894 14517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:21.133909 14517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:21.133922 14517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:21.133935 14517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:21.135516 14517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:33089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:21.135743 14517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:21.135932 14517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:21.141592 14525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:21.141567 14523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:21.141599 14517 server_base.cc:1034] running on GCE node
W20260810 06:38:21.141567 14527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:21.142144 14517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:21.142423 14517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:21.143577 14517 hybrid_clock.cc:648] HybridClock initialized: now 1786343901143562 us; error 33 us; skew 500 ppm
Aug 10 06:38:21 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:21.146492 14517 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:21.147117 14517 webserver.cc:460] Webserver started at http://127.30.71.62:40153/ using document root <none> and password file <none>
I20260810 06:38:21.147378 14517 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:21.147433 14517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:21.148942 14517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:38:21.149765 14535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.149947 14517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:21.150048 14517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "b871456921a547b4b807181e1084939b"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "d0a774b4801eb2179a743357ae625532"
server_key_iv: "c7d7301b95fa032af7dd1ea95d3cfa2a"
server_key_version: "encryptionkey@0"
I20260810 06:38:21.150429 14517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:21.178869 14517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:21.181594 14517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:21.181718 14517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:21.185837 14517 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:33089
I20260810 06:38:21.185916 14589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:33089 every 8 connection(s)
I20260810 06:38:21.186301 14517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:21.188035 14590 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:38:21.188783 14590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Bootstrap starting.
I20260810 06:38:21.191051 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 14517
I20260810 06:38:21.191376 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:38945
--local_ip_for_outbound_sockets=127.30.71.1
--tserver_master_addrs=127.30.71.62:33089
--webserver_port=43623
--webserver_interface=127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:21.198561 14590 log.cc:824] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:21.201041 14590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:21.201411 14590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b: Bootstrap complete.
I20260810 06:38:21.203890 14590 raft_consensus.cc:348] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:21.204128 14590 raft_consensus.cc:729] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b871456921a547b4b807181e1084939b, State: Initialized, Role: FOLLOWER
I20260810 06:38:21.204313 14590 consensus_queue.cc:260] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:21.204393 14590 raft_consensus.cc:388] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:21.204444 14590 raft_consensus.cc:482] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:21.204494 14590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:38:21.205575 14590 raft_consensus.cc:504] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:21.205705 14590 leader_election.cc:302] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b871456921a547b4b807181e1084939b; no voters:
I20260810 06:38:21.205945 14590 leader_election.cc:288] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:38:21.205982 14594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:38:21.206099 14594 raft_consensus.cc:686] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [term 3 LEADER]: Becoming Leader. State: Replica: b871456921a547b4b807181e1084939b, State: Running, Role: LEADER
I20260810 06:38:21.206249 14590 sys_catalog.cc:564] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:21.206283 14594 consensus_queue.cc:237] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } }
I20260810 06:38:21.206686 14595 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b871456921a547b4b807181e1084939b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } } }
I20260810 06:38:21.206722 14596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b871456921a547b4b807181e1084939b. Latest consensus state: current_term: 3 leader_uuid: "b871456921a547b4b807181e1084939b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871456921a547b4b807181e1084939b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33089 } } }
I20260810 06:38:21.206822 14596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:21.206777 14595 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871456921a547b4b807181e1084939b [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:21.207124 14601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:21.208081 14601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:21.208256 14601 catalog_manager.cc:1259] Loaded cluster ID: a34de976042c49ebb26ccad9ad69e9e6
I20260810 06:38:21.208505 14601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 06:38:21.208841 14601 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe7b51909ad google::LogMessage::Fail() at ??:0
@ 0x7fe7b519388a google::LogMessage::SendToLog() at ??:0
@ 0x7fe7b51904de google::LogMessage::Flush() at ??:0
@ 0x7fe7b5191a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe7bd0b10f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe7bd0ba8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe7b705e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe7b7053954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe7b7eb76db start_thread at ??:0
@ 0x7fe7b3c8e61f clone at ??:0
W20260810 06:38:21.293252 14592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:21.293483 14592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:21.293509 14592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:21.294721 14592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:21.294767 14592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:21.294795 14592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:38:21.296993 14592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:38945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=43623
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:21.297258 14592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:21.297549 14592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:21.302922 14634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:21.302953 14630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:21.302932 14631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:21.303257 14592 server_base.cc:1034] running on GCE node
I20260810 06:38:21.303488 14592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:21.303745 14592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:21.304912 14592 hybrid_clock.cc:648] HybridClock initialized: now 1786343901304897 us; error 32 us; skew 500 ppm
Aug 10 06:38:21 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:21.307667 14592 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:38:21.308182 14592 webserver.cc:460] Webserver started at http://127.30.71.1:43623/ using document root <none> and password file <none>
I20260810 06:38:21.308463 14592 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:21.308535 14592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:21.310401 14592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 06:38:21.311621 14641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.311879 14592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:38:21.312027 14592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "b8843ac6f70a4c2e804f69c52ea23203"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "270736b053570348cc87f60d17659f4e"
server_key_iv: "2bce2e9aa2ff8ddff45ffe8450cb3476"
server_key_version: "encryptionkey@0"
I20260810 06:38:21.312464 14592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:21.322937 14592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:21.325549 14592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:21.325721 14592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:21.325999 14592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:21.326489 14649 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:38:21.328089 14592 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:38:21.328145 14592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 06:38:21.328172 14592 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:38:21.328850 14592 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:38:21.328893 14592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 06:38:21.328949 14649 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap starting.
I20260810 06:38:21.336838 14592 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38945
I20260810 06:38:21.337042 14767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38945 every 8 connection(s)
I20260810 06:38:21.337411 14592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:21.338133 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 14592
I20260810 06:38:21.338454 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:45115
--local_ip_for_outbound_sockets=127.30.71.2
--tserver_master_addrs=127.30.71.62:33089
--webserver_port=36757
--webserver_interface=127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:21.341919 14649 log.cc:824] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:21.345011 14649 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:21.345382 14649 tablet_bootstrap.cc:492] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Bootstrap complete.
I20260810 06:38:21.346695 14649 ts_tablet_manager.cc:1397] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20260810 06:38:21.348284 14649 raft_consensus.cc:348] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:21.348621 14649 raft_consensus.cc:729] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8843ac6f70a4c2e804f69c52ea23203, State: Initialized, Role: FOLLOWER
I20260810 06:38:21.348790 14649 consensus_queue.cc:260] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:21.349107 14649 ts_tablet_manager.cc:1428] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260810 06:38:21.404110 14770 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:38:21.338177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:54157)
0810 06:38:21.338408 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.338461 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:21.339570 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:21.403889 (+ 64319us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":101,"threads_started":1}
W20260810 06:38:21.404443 14768 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:33089 (0 consecutive failures): Network error: Failed to ping master at 127.30.71.62:33089: Client connection negotiation failed: client connection to 127.30.71.62:33089: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:38:21.404842 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.404715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:34619)
0810 06:38:21.404758 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.404813 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 06:38:21.405203 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.405113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:54733)
0810 06:38:21.405156 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.405184 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 06:38:21.405366 14768 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:38:21.429841 14771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:21.430073 14771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:21.430101 14771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:21.431392 14771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:21.431439 14771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:21.431468 14771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:38:21.432997 14771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:45115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=36757
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:21.433259 14771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:21.433509 14771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:21.438822 14781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:21.438874 14782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:21.439002 14771 server_base.cc:1034] running on GCE node
W20260810 06:38:21.438843 14784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:21.439309 14771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:21.439581 14771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:21.440727 14771 hybrid_clock.cc:648] HybridClock initialized: now 1786343901440703 us; error 36 us; skew 500 ppm
Aug 10 06:38:21 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:21.448732 14771 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:38:21.449194 14771 webserver.cc:460] Webserver started at http://127.30.71.2:36757/ using document root <none> and password file <none>
I20260810 06:38:21.449407 14771 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:21.449451 14771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:21.450938 14771 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:21.451663 14791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.451838 14771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:21.451936 14771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "b04558c3273b41b9946fc0ffdadf7632"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "5012b409b7d52b6166657721415c764c"
server_key_iv: "b0234f6868046bc8cc68b8c8a368655b"
server_key_version: "encryptionkey@0"
I20260810 06:38:21.452282 14771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:21.468519 14771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:21.470487 14771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:21.470613 14771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:21.470842 14771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:21.471238 14771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:21.471277 14771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.471302 14771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:21.471314 14771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.477377 14771 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:45115
I20260810 06:38:21.477468 14916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:45115 every 8 connection(s)
I20260810 06:38:21.477819 14771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:21.478964 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.478547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:54609)
0810 06:38:21.478725 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.478807 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
W20260810 06:38:21.479424 14917 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:33089 (0 consecutive failures): Network error: Failed to ping master at 127.30.71.62:33089: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
I20260810 06:38:21.479719 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.479629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:50585)
0810 06:38:21.479675 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.479702 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 06:38:21.480036 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.479943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:43137)
0810 06:38:21.479989 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.480013 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 06:38:21.480139 14917 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:38:21.486305 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 14771
I20260810 06:38:21.486642 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:38107
--local_ip_for_outbound_sockets=127.30.71.3
--tserver_master_addrs=127.30.71.62:33089
--webserver_port=39709
--webserver_interface=127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:21.574995 14920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:21.575268 14920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:21.575313 14920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:21.576645 14920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:21.576745 14920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:21.576795 14920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:38:21.578401 14920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:38107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=39709
--tserver_master_addrs=127.30.71.62:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:21.578689 14920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:21.578941 14920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:21.584686 14928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:21.584703 14931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:21.584703 14927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:21.585081 14920 server_base.cc:1034] running on GCE node
I20260810 06:38:21.585302 14920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:21.585570 14920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:21.586715 14920 hybrid_clock.cc:648] HybridClock initialized: now 1786343901586696 us; error 36 us; skew 500 ppm
Aug 10 06:38:21 dist-test-slave-w206 krb5kdc[13330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:21.589593 14920 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:38:21.590083 14920 webserver.cc:460] Webserver started at http://127.30.71.3:39709/ using document root <none> and password file <none>
I20260810 06:38:21.590348 14920 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:21.590425 14920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:21.592033 14920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:21.592675 14940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.592841 14920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:38:21.592944 14920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "9b9a856fe00a4e219718092b71fc2db2"
format_stamp: "Formatted at 2026-08-10 06:38:19 on dist-test-slave-w206"
server_key: "572b69f81ec3631ea329b77ab1909599"
server_key_iv: "a55b883183e69587cb32abf8c8a0a3b3"
server_key_version: "encryptionkey@0"
I20260810 06:38:21.593317 14920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:21.613785 14920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:21.616695 14920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:21.616916 14920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:21.617208 14920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:21.617552 14920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:21.617611 14920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.617653 14920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:21.617682 14920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:21.623961 14920 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:38107
I20260810 06:38:21.624061 15068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:38107 every 8 connection(s)
I20260810 06:38:21.624441 14920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:21.625638 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.625230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:59209)
0810 06:38:21.625402 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.625492 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
W20260810 06:38:21.626148 15069 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:33089 (0 consecutive failures): Network error: Failed to ping master at 127.30.71.62:33089: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
I20260810 06:38:21.626515 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.626410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:51343)
0810 06:38:21.626464 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.626495 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 06:38:21.626828 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:21.626749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:50867)
0810 06:38:21.626791 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:21.626812 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 06:38:21.626924 15069 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:38:21.633041 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 14920
I20260810 06:38:22.406092 15073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:22.405805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:45733)
0810 06:38:22.405971 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:22.406020 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
I20260810 06:38:22.480752 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:22.480496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:49259)
0810 06:38:22.480645 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:22.480690 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
I20260810 06:38:22.627499 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:22.627262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:45165)
0810 06:38:22.627398 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:22.627441 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260810 06:38:23.122932 15076 raft_consensus.cc:482] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:23.123302 15076 raft_consensus.cc:504] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:23.123744 15076 leader_election.cc:288] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115), 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107)
Aug 10 06:38:23 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.2@KRBTEST.COM
Aug 10 06:38:23 dist-test-slave-w206 krb5kdc[13330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.3@KRBTEST.COM
I20260810 06:38:23.135397 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:23.124281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:45519 (local address 127.30.71.3:38107)
0810 06:38:23.124444 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:23.124455 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:23.124488 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:38:23.124969 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:23.124971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:23.125011 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:23.125173 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:23.125735 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.126740 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.127699 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.128141 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.128291 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:23.131314 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:23.131319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:23.131319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:23.131333 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:23.133301 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:23.134076 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:23.134079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:23.134080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:23.134199 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:23.134488 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:23.134489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:23.134489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:23.134750 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:23.134834 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:23.135141 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 06:38:23.135276 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260810 06:38:23.135434 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:23.123934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:45115 (local address 127.30.71.1:38495)
0810 06:38:23.124087 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:23.124102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:23.124168 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:23.125061 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:23.125064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:23.125411 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:23.125595 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:23.125600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.126698 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.126699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:23.127565 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:23.127567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.128239 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.128240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:23.128369 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:23.129018 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:23.129026 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:23.130759 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:23.133418 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:23.133426 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:23.133428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:23.133665 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:23.134130 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:23.134131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:23.134131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:23.134270 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:23.134757 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:23.134765 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:23.134912 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 06:38:23.135060 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 06:38:23.135276 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260810 06:38:23.135452 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:23.124067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:38495 (local address 127.30.71.2:45115)
0810 06:38:23.124270 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:23.124282 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:23.124306 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:38:23.124814 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:23.124817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:23.124866 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:23.125037 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:23.125719 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.126583 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.127664 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.128140 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.128267 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:23.130918 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:23.130923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:23.130924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:23.130938 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:23.133307 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:23.133780 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:23.133783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:23.133783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:23.133939 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:23.134346 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:23.134347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:23.134348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:23.134681 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:23.134756 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:23.135147 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 06:38:23.135325 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
I20260810 06:38:23.135730 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:23.124155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38107 (local address 127.30.71.1:45519)
0810 06:38:23.124316 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:23.124327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:23.124406 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:23.125165 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:23.125166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:23.125421 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:23.125596 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:23.125601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.126849 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.126850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:23.127565 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:23.127567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:23.128266 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:23.128267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:23.128364 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:23.129080 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:23.129086 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:23.131189 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:23.133781 (+ 2592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:23.133782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:23.133782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:23.133962 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:23.134322 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:23.134323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:23.134324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:23.134409 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:23.134842 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:23.134842 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:23.134912 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 06:38:23.135074 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 06:38:23.135369 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":143,"thread_start_us":97,"threads_started":1}
W20260810 06:38:23.136435 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
W20260810 06:38:23.136543 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
I20260810 06:38:23.136583 14643 leader_election.cc:302] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8843ac6f70a4c2e804f69c52ea23203; no voters: 9b9a856fe00a4e219718092b71fc2db2, b04558c3273b41b9946fc0ffdadf7632
I20260810 06:38:23.136687 15076 raft_consensus.cc:2726] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:38:23.406919 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:23.406712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:40355)
0810 06:38:23.406817 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:23.406871 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:38:23.481582 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:23.481288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:49169)
0810 06:38:23.481458 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:23.481524 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260810 06:38:23.628412 15084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:23.628157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:46273)
0810 06:38:23.628304 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:23.628352 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260810 06:38:24.408020 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:24.407656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:34169)
0810 06:38:24.407904 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:24.407955 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":73,"threads_started":1}
I20260810 06:38:24.482448 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:24.482180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:49359)
0810 06:38:24.482348 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:24.482398 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":64,"threads_started":1}
I20260810 06:38:24.629194 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:24.628936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:44725)
0810 06:38:24.629101 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:24.629145 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260810 06:38:25.216585 15090 raft_consensus.cc:482] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:25.216722 15090 raft_consensus.cc:504] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:25.216935 15090 leader_election.cc:288] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115), 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107)
W20260810 06:38:25.217650 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
W20260810 06:38:25.217726 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
I20260810 06:38:25.217744 14643 leader_election.cc:302] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8843ac6f70a4c2e804f69c52ea23203; no voters: 9b9a856fe00a4e219718092b71fc2db2, b04558c3273b41b9946fc0ffdadf7632
I20260810 06:38:25.217868 15090 raft_consensus.cc:2726] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:38:25.409230 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:25.408942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:54447)
0810 06:38:25.409114 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:25.409164 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260810 06:38:25.483330 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:25.483037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:54193)
0810 06:38:25.483193 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:25.483246 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260810 06:38:25.630050 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:25.629717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:60367)
0810 06:38:25.629905 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:25.629958 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260810 06:38:26.410127 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.409852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:53403)
0810 06:38:26.410019 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:26.410076 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260810 06:38:26.484119 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.483893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:54329)
0810 06:38:26.484027 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:26.484066 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260810 06:38:26.631045 15097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:26.630735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:58465)
0810 06:38:26.630894 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:26.630948 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260810 06:38:26.935866 15098 raft_consensus.cc:482] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:26.935994 15098 raft_consensus.cc:504] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:26.936251 15098 leader_election.cc:288] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115), 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107)
W20260810 06:38:26.936723 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
W20260810 06:38:26.936816 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
I20260810 06:38:26.936854 14643 leader_election.cc:302] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8843ac6f70a4c2e804f69c52ea23203; no voters: 9b9a856fe00a4e219718092b71fc2db2, b04558c3273b41b9946fc0ffdadf7632
I20260810 06:38:26.936961 15098 raft_consensus.cc:2726] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:38:27.410933 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:27.410692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:41409)
0810 06:38:27.410841 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:27.410878 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260810 06:38:27.484894 15100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:27.484641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:43063)
0810 06:38:27.484797 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:27.484840 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260810 06:38:27.631990 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:27.631706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:47777)
0810 06:38:27.631878 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:27.631940 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
I20260810 06:38:28.411789 15102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:28.411536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:48805)
0810 06:38:28.411690 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:28.411736 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260810 06:38:28.485697 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:28.485436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:34121)
0810 06:38:28.485589 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:28.485637 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260810 06:38:28.632893 15104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:28.632588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:47829)
0810 06:38:28.632780 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:28.632826 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260810 06:38:29.412626 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.412340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:39457)
0810 06:38:29.412525 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.412576 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260810 06:38:29.486543 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.486217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:37607)
0810 06:38:29.486436 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.486490 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
I20260810 06:38:29.633718 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:29.633460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:54049)
0810 06:38:29.633621 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:29.633667 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260810 06:38:30.413604 15108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:30.413310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:60493)
0810 06:38:30.413477 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:30.413520 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
I20260810 06:38:30.487561 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:30.487283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:37483)
0810 06:38:30.487459 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:30.487508 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260810 06:38:30.569329 15110 raft_consensus.cc:482] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:30.569442 15110 raft_consensus.cc:504] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:30.569622 15110 leader_election.cc:288] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115), 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107)
W20260810 06:38:30.570062 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
W20260810 06:38:30.570132 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
I20260810 06:38:30.570147 14643 leader_election.cc:302] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8843ac6f70a4c2e804f69c52ea23203; no voters: 9b9a856fe00a4e219718092b71fc2db2, b04558c3273b41b9946fc0ffdadf7632
I20260810 06:38:30.570225 15110 raft_consensus.cc:2726] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:38:30.634598 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:30.634378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:41033)
0810 06:38:30.634500 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:30.634547 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260810 06:38:31.414467 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:31.414125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:51165)
0810 06:38:31.414352 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:31.414395 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":63,"threads_started":1}
I20260810 06:38:31.488466 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:31.488182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:52605)
0810 06:38:31.488349 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:31.488408 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260810 06:38:31.635557 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:31.635292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:37683)
0810 06:38:31.635460 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:31.635506 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260810 06:38:32.415330 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:32.415077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:41409)
0810 06:38:32.415220 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:32.415280 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260810 06:38:32.489246 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:32.488963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:39193)
0810 06:38:32.489138 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:32.489185 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":66,"threads_started":1}
I20260810 06:38:32.636281 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:32.636023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:36123)
0810 06:38:32.636180 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:32.636220 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260810 06:38:33.416082 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:33.415856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:54797)
0810 06:38:33.415989 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:33.416034 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260810 06:38:33.490054 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:33.489760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:35961)
0810 06:38:33.489917 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:33.489984 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
I20260810 06:38:33.637172 15121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:33.636933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:42665)
0810 06:38:33.637079 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:33.637122 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260810 06:38:34.416855 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:34.416612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:53271)
0810 06:38:34.416754 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:34.416802 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260810 06:38:34.490887 15123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:34.490614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:59503)
0810 06:38:34.490763 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:34.490812 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260810 06:38:34.638139 15124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:34.637882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:41289)
0810 06:38:34.638045 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:34.638091 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260810 06:38:34.992560 15125 raft_consensus.cc:482] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:34.992705 15125 raft_consensus.cc:504] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b04558c3273b41b9946fc0ffdadf7632" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 45115 } } peers { permanent_uuid: "9b9a856fe00a4e219718092b71fc2db2" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38107 } } peers { permanent_uuid: "b8843ac6f70a4c2e804f69c52ea23203" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38945 } }
I20260810 06:38:34.992906 15125 leader_election.cc:288] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115), 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107)
W20260810 06:38:34.993373 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b04558c3273b41b9946fc0ffdadf7632 (127.30.71.2:45115): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
W20260810 06:38:34.993451 14643 leader_election.cc:341] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9b9a856fe00a4e219718092b71fc2db2 (127.30.71.3:38107): Not found: Tablet not found: 1c532d0fda28419990ee101796d8a45d
I20260810 06:38:34.993479 14643 leader_election.cc:302] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8843ac6f70a4c2e804f69c52ea23203; no voters: 9b9a856fe00a4e219718092b71fc2db2, b04558c3273b41b9946fc0ffdadf7632
I20260810 06:38:34.993580 15125 raft_consensus.cc:2726] T 1c532d0fda28419990ee101796d8a45d P b8843ac6f70a4c2e804f69c52ea23203 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:38:35.417580 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:35.417348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:33733)
0810 06:38:35.417489 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:35.417531 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260810 06:38:35.491695 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:35.491444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:40483)
0810 06:38:35.491578 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:35.491642 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260810 06:38:35.639053 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:35.638709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:52193)
0810 06:38:35.638886 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:35.638934 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260810 06:38:36.418447 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:36.418194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.1:43699)
0810 06:38:36.418356 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:36.418401 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260810 06:38:36.492538 15130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:36.492319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.2:49851)
0810 06:38:36.492446 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:36.492486 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260810 06:38:36.635767 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 14592
I20260810 06:38:36.639806 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:36.639593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:33089 (local address 127.30.71.3:35789)
0810 06:38:36.639724 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:36.639763 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260810 06:38:36.642304 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 14771
I20260810 06:38:36.647969 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 14920
2026-08-10T06:38:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18818 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:38:36 dist-test-slave-w206 krb5kdc[15134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:38:36 dist-test-slave-w206 krb5kdc[15134](info): set up 2 sockets
Aug 10 06:38:36 dist-test-slave-w206 krb5kdc[15134](info): commencing operation
krb5kdc: starting...
W20260810 06:38:37.734747 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:38:37 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:38:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:38:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:38:37.867465 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43125
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:37.951730 15148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:37.952016 15148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:37.952056 15148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:37.953284 15148 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:38:37.953362 15148 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:38:37.953383 15148 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:38:37.953403 15148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:37.953434 15148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:37.953464 15148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:37.953490 15148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:37.953521 15148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:37.955091 15148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:37.955399 15148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:37.955657 15148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:37.961330 15153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:37.961362 15154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:37.961377 15156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:37.961397 15148 server_base.cc:1034] running on GCE node
I20260810 06:38:37.961841 15148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:37.962142 15148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:37.963341 15148 hybrid_clock.cc:648] HybridClock initialized: now 1786343917963334 us; error 43 us; skew 500 ppm
Aug 10 06:38:37 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:37.966032 15148 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:37.966589 15148 webserver.cc:460] Webserver started at http://127.30.71.62:43977/ using document root <none> and password file <none>
I20260810 06:38:37.966825 15148 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:37.966895 15148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:37.967079 15148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:37.968225 15148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "cfac2f610f094886a954296e2b7186cf"
format_stamp: "Formatted at 2026-08-10 06:38:37 on dist-test-slave-w206"
server_key: "79a9a4ddba25ba3774fba63c281838e4"
server_key_iv: "f1196a74415bed1b28b3d54c46a91bde"
server_key_version: "encryptionkey@0"
I20260810 06:38:37.968612 15148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "cfac2f610f094886a954296e2b7186cf"
format_stamp: "Formatted at 2026-08-10 06:38:37 on dist-test-slave-w206"
server_key: "79a9a4ddba25ba3774fba63c281838e4"
server_key_iv: "f1196a74415bed1b28b3d54c46a91bde"
server_key_version: "encryptionkey@0"
I20260810 06:38:37.970098 15148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:38:37.970918 15163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:37.971120 15148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:37.971227 15148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "cfac2f610f094886a954296e2b7186cf"
format_stamp: "Formatted at 2026-08-10 06:38:37 on dist-test-slave-w206"
server_key: "79a9a4ddba25ba3774fba63c281838e4"
server_key_iv: "f1196a74415bed1b28b3d54c46a91bde"
server_key_version: "encryptionkey@0"
I20260810 06:38:37.971339 15148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:37.989399 15148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:37.991998 15148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:37.992178 15148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:37.996666 15148 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:43125
I20260810 06:38:37.996733 15220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:43125 every 8 connection(s)
I20260810 06:38:37.997110 15148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:37.998137 15221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:38.000789 15221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Bootstrap starting.
I20260810 06:38:38.001530 15221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:38.001816 15221 log.cc:824] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:38.002575 15221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: No bootstrap required, opened a new log
I20260810 06:38:38.003674 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 15148
I20260810 06:38:38.003815 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:38:38.004048 31004 external_mini_cluster.cc:1428] Setting key 53838ef7900f901d5ed18c16023212ce
I20260810 06:38:38.005180 15221 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:38.005366 15221 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:38.005409 15221 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfac2f610f094886a954296e2b7186cf, State: Initialized, Role: FOLLOWER
I20260810 06:38:38.005615 15221 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:38.005722 15221 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:38.005784 15221 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:38.005846 15221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:38.006671 15221 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:38.006826 15221 leader_election.cc:302] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfac2f610f094886a954296e2b7186cf; no voters:
I20260810 06:38:38.007076 15221 leader_election.cc:288] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:38:38.007140 15226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:38.007301 15226 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 LEADER]: Becoming Leader. State: Replica: cfac2f610f094886a954296e2b7186cf, State: Running, Role: LEADER
I20260810 06:38:38.007383 15221 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:38.007472 15226 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:38.008005 15227 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfac2f610f094886a954296e2b7186cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } } }
I20260810 06:38:38.008098 15227 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:38.008272 15229 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfac2f610f094886a954296e2b7186cf. Latest consensus state: current_term: 1 leader_uuid: "cfac2f610f094886a954296e2b7186cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } } }
I20260810 06:38:38.008322 15229 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: This master's current role is: LEADER
W20260810 06:38:38.009591 15243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:38:38.009658 15243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:38:38.009716 15235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:38.009933 15235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:38.012615 15235 catalog_manager.cc:1347] Generated new cluster ID: 5edd772b1e724df89376537a4914969f
I20260810 06:38:38.012686 15235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:38:38 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:38.017530 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.004653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.30.71.62:43125)
0810 06:38:38.004842 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.004852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.004887 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:38:38.005150 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.005156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.005204 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.005356 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.006267 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.008762 (+ 2495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.009801 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.010367 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.010501 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.013575 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:38.013579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:38.013580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:38.013594 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:38.015641 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.016121 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.016125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.016125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.016263 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.016504 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.016504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.016505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.016837 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:38.016974 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.017138 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.017292 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":124,"threads_started":1}
I20260810 06:38:38.033927 15235 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:38:38.034788 15235 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:38.040195 15235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Generated new TSK 0
I20260810 06:38:38.040449 15235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:38:38.071399 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43125
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:38.162904 15248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:38.163205 15248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:38.163235 15248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:38.164419 15248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:38.164466 15248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:38.164494 15248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:38:38.165995 15248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:38.166210 15248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:38.166522 15248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:38.172067 15256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:38.172070 15254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:38.172061 15253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:38.172443 15248 server_base.cc:1034] running on GCE node
I20260810 06:38:38.172654 15248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:38.172887 15248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:38.174083 15248 hybrid_clock.cc:648] HybridClock initialized: now 1786343918174070 us; error 36 us; skew 500 ppm
Aug 10 06:38:38 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:38.177057 15248 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:38:38.177592 15248 webserver.cc:460] Webserver started at http://127.30.71.1:38561/ using document root <none> and password file <none>
I20260810 06:38:38.177842 15248 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:38.177920 15248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:38.178071 15248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:38.179296 15248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "51ad1c6a7c114c138361eaae7cdffbdc"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "69c89089cf632201256f29b7e442fe66"
server_key_iv: "90997936ee013091044eeafe030f38a6"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.179675 15248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "51ad1c6a7c114c138361eaae7cdffbdc"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "69c89089cf632201256f29b7e442fe66"
server_key_iv: "90997936ee013091044eeafe030f38a6"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.181197 15248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:38:38.182032 15263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.182219 15248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:38.182330 15248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "51ad1c6a7c114c138361eaae7cdffbdc"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "69c89089cf632201256f29b7e442fe66"
server_key_iv: "90997936ee013091044eeafe030f38a6"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.182411 15248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:38.195103 15248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:38.197675 15248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:38.197860 15248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:38.198123 15248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:38.198473 15248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:38.198535 15248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.198576 15248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:38.198603 15248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.204828 15248 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:42841
I20260810 06:38:38.204849 15394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:42841 every 8 connection(s)
I20260810 06:38:38.205224 15248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:38.207840 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 15248
I20260810 06:38:38.207975 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:38:38.208241 31004 external_mini_cluster.cc:1428] Setting key 43e2baa3e549082b0f45039dce68d44c
Aug 10 06:38:38 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:38.217141 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.206012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:46985 (local address 127.30.71.62:43125)
0810 06:38:38.206085 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.206087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.206420 (+ 333us) server_negotiation.cc:407] Connection header received
0810 06:38:38.206902 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.206903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.206914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.207038 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.208276 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.209021 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.210632 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.211040 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.211121 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.213759 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:38.213761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:38.213762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:38.213771 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:38.215464 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.216005 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.216006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.216006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.216066 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.216390 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.216391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.216391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.216508 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:38.216558 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.216969 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.217069 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:38:38.217427 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.205987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:46985)
0810 06:38:38.206259 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.206300 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.206811 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.207119 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.207123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.207866 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:38.208173 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.208181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.209149 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.209150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.210522 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.210527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.211136 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.211137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.211255 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.211861 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:38.211869 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:38.213638 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:38.215556 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:38.215566 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:38.215569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:38.215912 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:38.216143 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:38.216143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:38.216144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:38.216319 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:38.216586 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:38.216593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:38.216784 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.216918 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.217128 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:38:38.218171 15395 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43125
I20260810 06:38:38.218298 15395 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:38.218484 15395 heartbeater.cc:511] Master 127.30.71.62:43125 requested a full tablet report, sending...
I20260810 06:38:38.219192 15180 ts_manager.cc:194] Registered new tserver with Master: 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1:42841)
I20260810 06:38:38.219805 15180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:46985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:38:38.238535 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43125
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:38.329038 15401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:38.329298 15401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:38.329325 15401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:38.330464 15401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:38.330503 15401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:38.330531 15401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:38:38.331985 15401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:38.332193 15401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:38.332392 15401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:38.337749 15407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:38.337756 15408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:38.338004 15401 server_base.cc:1034] running on GCE node
W20260810 06:38:38.337770 15410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:38.338321 15401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:38.338636 15401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:38.339831 15401 hybrid_clock.cc:648] HybridClock initialized: now 1786343918339805 us; error 33 us; skew 500 ppm
Aug 10 06:38:38 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:38.342682 15401 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:38:38.343281 15401 webserver.cc:460] Webserver started at http://127.30.71.2:33003/ using document root <none> and password file <none>
I20260810 06:38:38.343538 15401 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:38.343590 15401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:38.343753 15401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:38.344962 15401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "074b579872b842a385332ca1032ffc26"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "41084789244c9c8e749398cd074b562b"
server_key_iv: "1ea0865dcafef6720ec62463c8fcbfe4"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.345363 15401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "074b579872b842a385332ca1032ffc26"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "41084789244c9c8e749398cd074b562b"
server_key_iv: "1ea0865dcafef6720ec62463c8fcbfe4"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.346860 15401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:38:38.347702 15418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.347950 15401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:38.348058 15401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "074b579872b842a385332ca1032ffc26"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "41084789244c9c8e749398cd074b562b"
server_key_iv: "1ea0865dcafef6720ec62463c8fcbfe4"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.348145 15401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:38.363787 15401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:38.366118 15401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:38.366305 15401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:38.366616 15401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:38.367057 15401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:38.367111 15401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.367137 15401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:38.367151 15401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.374295 15401 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:43449
I20260810 06:38:38.374341 15543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:43449 every 8 connection(s)
I20260810 06:38:38.374766 15401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:38:38.375360 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 15401
I20260810 06:38:38.375494 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:38:38.375736 31004 external_mini_cluster.cc:1428] Setting key 6b226da30e66b6a45eb9b2e72d617c01
Aug 10 06:38:38 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
I20260810 06:38:38.385666 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.375597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:56715 (local address 127.30.71.62:43125)
0810 06:38:38.375653 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.375655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.375954 (+ 299us) server_negotiation.cc:407] Connection header received
0810 06:38:38.376489 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.376490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.376502 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.376550 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.377557 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.378197 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.379429 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.379763 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.379852 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.382286 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:38.382288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:38.382288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:38.382298 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:38.384058 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.384546 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.384548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.384548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.384613 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.384940 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.384941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.384942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.385105 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:38.385166 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.385496 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.385569 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:38:38.385999 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.375595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:56715)
0810 06:38:38.375816 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.375869 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.376385 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.376620 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.376624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.377237 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:38.377448 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.377453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.378301 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.378302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.379340 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.379343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.379858 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.379859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.379980 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.380661 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:38.380671 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:38.382179 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:38.384154 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:38.384161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:38.384163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:38.384463 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:38.384689 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:38.384690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:38.384690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:38.384868 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:38.385173 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:38.385178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:38.385342 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.385457 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.385664 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260810 06:38:38.386710 15544 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43125
I20260810 06:38:38.386832 15544 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:38.387094 15544 heartbeater.cc:511] Master 127.30.71.62:43125 requested a full tablet report, sending...
I20260810 06:38:38.387526 15180 ts_manager.cc:194] Registered new tserver with Master: 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
I20260810 06:38:38.388025 15180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:56715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:38:38.403995 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43125
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:38.488543 15550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:38.488770 15550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:38.488796 15550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:38.489996 15550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:38.490042 15550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:38.490070 15550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:38:38.491590 15550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:38.491850 15550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:38.492105 15550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:38.497572 15558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:38.497537 15555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:38.497510 15556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:38.497575 15550 server_base.cc:1034] running on GCE node
I20260810 06:38:38.498039 15550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:38.498301 15550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:38.499462 15550 hybrid_clock.cc:648] HybridClock initialized: now 1786343918499458 us; error 38 us; skew 500 ppm
Aug 10 06:38:38 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:38.502022 15550 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:38:38.502677 15550 webserver.cc:460] Webserver started at http://127.30.71.3:41127/ using document root <none> and password file <none>
I20260810 06:38:38.502921 15550 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:38.503012 15550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:38.503173 15550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:38:38.504318 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "a66ea2fb841b4011a0eaa27be6e8f920"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "9c2c4fa7ec931b8696d6b718af6cfcf4"
server_key_iv: "23c1d5de90f6d9d8a57211919a37ee01"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.504686 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "a66ea2fb841b4011a0eaa27be6e8f920"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "9c2c4fa7ec931b8696d6b718af6cfcf4"
server_key_iv: "23c1d5de90f6d9d8a57211919a37ee01"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.506060 15550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:38:38.506857 15566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.507078 15550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:38:38.507184 15550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "a66ea2fb841b4011a0eaa27be6e8f920"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "9c2c4fa7ec931b8696d6b718af6cfcf4"
server_key_iv: "23c1d5de90f6d9d8a57211919a37ee01"
server_key_version: "encryptionkey@0"
I20260810 06:38:38.507270 15550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:38.531899 15550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:38.534121 15550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:38.534303 15550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:38.534605 15550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:38.534956 15550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:38.535073 15550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.535121 15550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:38.535149 15550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:38.541642 15550 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:39053
I20260810 06:38:38.541687 15696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:39053 every 8 connection(s)
I20260810 06:38:38.542025 15550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:38:38 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:38.550025 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 15550
I20260810 06:38:38.550135 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:38:38.550336 31004 external_mini_cluster.cc:1428] Setting key b606658dc6b931acbcfc9d328546d6de
I20260810 06:38:38.551481 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.542820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:60689 (local address 127.30.71.62:43125)
0810 06:38:38.542890 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.542892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.543066 (+ 174us) server_negotiation.cc:407] Connection header received
0810 06:38:38.543620 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.543621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.543634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.543715 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.544655 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.545155 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.546110 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.546330 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.546383 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.548553 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:38.548555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:38.548555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:38.548562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:38.549760 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.550272 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.550273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.550274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.550345 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.550680 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.550682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.550682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.550842 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:38.550917 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.551249 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.551341 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:38:38.551694 15700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.542740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:60689)
0810 06:38:38.542915 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.542959 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.543495 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.543772 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.543775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.544364 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:38.544562 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.544567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.545241 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.545242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.546038 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.546039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.546405 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.546406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.546507 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.547042 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:38.547057 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:38.548465 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:38.549845 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:38.549862 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:38.549864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:38.550191 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:38.550456 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:38.550457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:38.550457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:38.550582 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:38.550921 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:38.550929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:38.551117 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.551206 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.551460 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260810 06:38:38.552186 15698 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43125
I20260810 06:38:38.552306 15698 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:38.552520 15698 heartbeater.cc:511] Master 127.30.71.62:43125 requested a full tablet report, sending...
I20260810 06:38:38.552918 15180 ts_manager.cc:194] Registered new tserver with Master: a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
I20260810 06:38:38.553290 15180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:60689
I20260810 06:38:38.562368 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:38.571501 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.563893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.30.71.62:43125)
0810 06:38:38.563984 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.563986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.564087 (+ 101us) server_negotiation.cc:407] Connection header received
0810 06:38:38.564198 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.564199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.564209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.564314 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:38.565036 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.565901 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.567075 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.567338 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.567431 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.568766 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:38.568768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:38.568768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:38.568775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:38.570165 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.570548 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.570550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.570550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.570609 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:38.570832 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:38.570833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:38.570833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:38.571072 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:38.571187 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.571310 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.571393 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:38:38.573055 15180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:38.574935 15180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:38.582309 15711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.577528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42841 (local address 127.0.0.1:52092)
0810 06:38:38.577716 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.577758 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.577829 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.578673 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.578674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.578676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:38.578723 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.578724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.579975 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.579976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.581059 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.581061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.581991 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.581993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.582067 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.582078 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.582124 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.582212 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260810 06:38:38.582701 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.577739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.30.71.1:42841)
0810 06:38:38.578493 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.578499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.578512 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:38:38.578542 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.578543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.578575 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.578661 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:38.578781 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.579856 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.581153 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.581893 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.582463 (+ 570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.582537 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.582569 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.582613 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":144,"threads_started":1}
I20260810 06:38:38.583174 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.577543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39053 (local address 127.0.0.1:60112)
0810 06:38:38.577978 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.577986 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.578023 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.578440 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.578443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.578446 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:38.578522 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.578524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.581161 (+ 2637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.581162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.581814 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.581815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.582874 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.582875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.582996 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.583001 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.583060 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.583111 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":115,"threads_started":1}
I20260810 06:38:38.583469 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.577946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.30.71.3:39053)
0810 06:38:38.578207 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.578211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.578221 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:38:38.578256 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.578257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.578281 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.578361 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:38.578598 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.579850 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.581901 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.582695 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.582774 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.582832 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.583339 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.583387 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":119,"threads_started":1}
I20260810 06:38:38.583557 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.577743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.30.71.2:43449)
0810 06:38:38.578188 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.578195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.578207 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:38:38.578252 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.578254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.578269 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.578361 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:38.578800 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.579547 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.581089 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.581893 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.581974 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.582039 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.583415 (+ 1376us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.583469 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":99,"threads_started":1}
I20260810 06:38:38.583475 15315 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1920f0b0a14ebab55db94bc1843067 (DEFAULT_TABLE table=test-table [id=7fd02cc4596f4e628522b63fb14ff0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:38.583360 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.577528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43449 (local address 127.0.0.1:40994)
0810 06:38:38.577803 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.577815 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.577871 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.578583 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.578584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.578585 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:38.578606 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.578606 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.579643 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.579644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.580998 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.581001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.581973 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.581974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.582057 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.582073 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.583244 (+ 1171us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.583296 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":46,"spinlock_wait_cycles":768,"thread_start_us":111,"threads_started":1}
I20260810 06:38:38.583894 15315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f1920f0b0a14ebab55db94bc1843067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:38.584154 15618 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1920f0b0a14ebab55db94bc1843067 (DEFAULT_TABLE table=test-table [id=7fd02cc4596f4e628522b63fb14ff0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:38.584406 15618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f1920f0b0a14ebab55db94bc1843067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:38.586854 15718 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap starting.
I20260810 06:38:38.587096 15719 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Bootstrap starting.
I20260810 06:38:38.587764 15718 tablet_bootstrap.cc:654] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:38.587777 15719 tablet_bootstrap.cc:654] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:38.588125 15472 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1920f0b0a14ebab55db94bc1843067 (DEFAULT_TABLE table=test-table [id=7fd02cc4596f4e628522b63fb14ff0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:38.588320 15718 log.cc:824] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:38.588358 15719 log.cc:824] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:38.588518 15472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f1920f0b0a14ebab55db94bc1843067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:38.589207 15718 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: No bootstrap required, opened a new log
I20260810 06:38:38.589296 15718 ts_tablet_manager.cc:1397] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:38.589412 15719 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: No bootstrap required, opened a new log
I20260810 06:38:38.589476 15719 ts_tablet_manager.cc:1397] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:38.591576 15722 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Bootstrap starting.
I20260810 06:38:38.591471 15718 raft_consensus.cc:348] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.591504 15719 raft_consensus.cc:348] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.591660 15718 raft_consensus.cc:374] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:38.591658 15719 raft_consensus.cc:374] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:38.591706 15719 raft_consensus.cc:729] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66ea2fb841b4011a0eaa27be6e8f920, State: Initialized, Role: FOLLOWER
I20260810 06:38:38.591734 15718 raft_consensus.cc:729] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad1c6a7c114c138361eaae7cdffbdc, State: Initialized, Role: FOLLOWER
I20260810 06:38:38.591847 15719 consensus_queue.cc:260] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.591847 15718 consensus_queue.cc:260] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.592149 15719 ts_tablet_manager.cc:1428] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:38.592207 15698 heartbeater.cc:503] Master 127.30.71.62:43125 was elected leader, sending a full tablet report...
I20260810 06:38:38.592149 15718 ts_tablet_manager.cc:1428] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:38:38.592448 15722 tablet_bootstrap.cc:654] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:38.592321 15395 heartbeater.cc:503] Master 127.30.71.62:43125 was elected leader, sending a full tablet report...
I20260810 06:38:38.593102 15722 log.cc:824] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:38.594033 15722 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: No bootstrap required, opened a new log
I20260810 06:38:38.594089 15722 ts_tablet_manager.cc:1397] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:38:38.595590 15722 raft_consensus.cc:348] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.595732 15722 raft_consensus.cc:374] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:38.595757 15722 raft_consensus.cc:729] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074b579872b842a385332ca1032ffc26, State: Initialized, Role: FOLLOWER
I20260810 06:38:38.595887 15722 consensus_queue.cc:260] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.596192 15722 ts_tablet_manager.cc:1428] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:38:38.596351 15544 heartbeater.cc:503] Master 127.30.71.62:43125 was elected leader, sending a full tablet report...
W20260810 06:38:38.625416 15545 tablet.cc:2369] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:38.706035 15396 tablet.cc:2369] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:38:38.792603 15699 tablet.cc:2369] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:38.907532 15724 raft_consensus.cc:482] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:38.907673 15724 raft_consensus.cc:504] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.908195 15724 leader_election.cc:288] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
I20260810 06:38:38.911550 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.908411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39053 (local address 127.30.71.1:60381)
0810 06:38:38.908555 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.908566 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.908641 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.908800 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.908801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.908803 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:38.908892 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.908895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.909644 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.909645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.910545 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.910548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.911243 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.911244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.911326 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.911330 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.911408 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.911453 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260810 06:38:38.911540 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.908513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:60381 (local address 127.30.71.3:39053)
0810 06:38:38.908560 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.908563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.908628 (+ 65us) server_negotiation.cc:407] Connection header received
0810 06:38:38.908714 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.908715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.908727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.908782 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:38.908989 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.909566 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.910651 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.911147 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.911240 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.911251 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.911415 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.911453 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:38:38.911631 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.908551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:60585 (local address 127.30.71.2:43449)
0810 06:38:38.908600 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.908602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.908633 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:38:38.908723 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.908724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.908737 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.908825 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:38.909035 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.909511 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.910757 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.911198 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.911350 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.911361 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.911526 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.911565 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 06:38:38.911649 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.908411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43449 (local address 127.30.71.1:60585)
0810 06:38:38.908553 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.908566 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:38.908652 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:38.908853 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:38.908854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:38.908856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:38.908967 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.908970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.909601 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.909602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.910665 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:38.910668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.911359 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.911360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:38.911436 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.911440 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:38:38.911519 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:38:38.911561 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":65,"thread_start_us":67,"threads_started":1}
I20260810 06:38:38.912312 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1920f0b0a14ebab55db94bc1843067" candidate_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074b579872b842a385332ca1032ffc26" is_pre_election: true
I20260810 06:38:38.912312 15647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1920f0b0a14ebab55db94bc1843067" candidate_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" is_pre_election: true
I20260810 06:38:38.912480 15495 raft_consensus.cc:2454] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51ad1c6a7c114c138361eaae7cdffbdc in term 0.
I20260810 06:38:38.912480 15647 raft_consensus.cc:2454] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51ad1c6a7c114c138361eaae7cdffbdc in term 0.
I20260810 06:38:38.912771 15267 leader_election.cc:302] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074b579872b842a385332ca1032ffc26, 51ad1c6a7c114c138361eaae7cdffbdc; no voters:
I20260810 06:38:38.912992 15724 raft_consensus.cc:2781] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:38.913035 15724 raft_consensus.cc:482] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:38.913053 15724 raft_consensus.cc:3037] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:38.913805 15724 raft_consensus.cc:504] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.913961 15724 leader_election.cc:288] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 1 election: Requested vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
I20260810 06:38:38.914122 15647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1920f0b0a14ebab55db94bc1843067" candidate_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a66ea2fb841b4011a0eaa27be6e8f920"
I20260810 06:38:38.914197 15647 raft_consensus.cc:3037] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:38.914153 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f1920f0b0a14ebab55db94bc1843067" candidate_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074b579872b842a385332ca1032ffc26"
I20260810 06:38:38.914258 15495 raft_consensus.cc:3037] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:38.914948 15647 raft_consensus.cc:2454] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51ad1c6a7c114c138361eaae7cdffbdc in term 1.
I20260810 06:38:38.915095 15495 raft_consensus.cc:2454] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51ad1c6a7c114c138361eaae7cdffbdc in term 1.
I20260810 06:38:38.915155 15268 leader_election.cc:302] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ad1c6a7c114c138361eaae7cdffbdc, a66ea2fb841b4011a0eaa27be6e8f920; no voters:
I20260810 06:38:38.915247 15724 raft_consensus.cc:2781] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:38.915444 15724 raft_consensus.cc:686] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 LEADER]: Becoming Leader. State: Replica: 51ad1c6a7c114c138361eaae7cdffbdc, State: Running, Role: LEADER
I20260810 06:38:38.915551 15724 consensus_queue.cc:237] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:38.916461 15179 catalog_manager.cc:5495] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1). New cstate: current_term: 1 leader_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:38.940796 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.938485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.30.71.1:42841)
0810 06:38:38.938555 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:38.938557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:38.938565 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:38:38.938613 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:38.938614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:38.938637 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:38.938705 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:38.938839 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.939370 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.940119 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:38.940314 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:38.940368 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:38.940509 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:38.940615 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:38.940676 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:38.940709 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 06:38:38.940727 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:38:38.942641 15647 raft_consensus.cc:1261] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 FOLLOWER]: Refusing update from remote peer 51ad1c6a7c114c138361eaae7cdffbdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:38.942639 15495 raft_consensus.cc:1261] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Refusing update from remote peer 51ad1c6a7c114c138361eaae7cdffbdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:38.943003 15724 consensus_queue.cc:1035] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:38.943167 15730 consensus_queue.cc:1035] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:38.945492 15733 mvcc.cc:205] Tried to move back new op lower bound from 7316864691986087936 to 7316864691878621184. Current Snapshot: MvccSnapshot[applied={T|T < 7316864691986087936}]
I20260810 06:38:38.948254 15180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:38.948369 15180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:38.949134 15180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Sending DeleteTablet for 3 replicas of tablet 2f1920f0b0a14ebab55db94bc1843067
I20260810 06:38:38.949503 15472 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f1920f0b0a14ebab55db94bc1843067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:38 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:40994
I20260810 06:38:38.949519 15618 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f1920f0b0a14ebab55db94bc1843067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:38 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:60112
I20260810 06:38:38.949764 15315 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f1920f0b0a14ebab55db94bc1843067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:38 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:52092
I20260810 06:38:38.949985 15746 tablet_replica.cc:330] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: stopping tablet replica
I20260810 06:38:38.950008 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 15248
I20260810 06:38:38.952204 15745 tablet_replica.cc:330] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: stopping tablet replica
I20260810 06:38:38.952358 15745 raft_consensus.cc:2229] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:38.952486 15745 raft_consensus.cc:2258] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:38.952739 15743 tablet_replica.cc:330] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: stopping tablet replica
I20260810 06:38:38.952894 15743 raft_consensus.cc:2229] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:38.953017 15745 ts_tablet_manager.cc:1905] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:38.953048 15743 raft_consensus.cc:2258] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:38.954958 15745 ts_tablet_manager.cc:1918] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:38.955092 15745 log.cc:1196] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/2f1920f0b0a14ebab55db94bc1843067
I20260810 06:38:38.955428 15745 ts_tablet_manager.cc:1939] T 2f1920f0b0a14ebab55db94bc1843067 P a66ea2fb841b4011a0eaa27be6e8f920: Deleting consensus metadata
I20260810 06:38:38.955447 15743 ts_tablet_manager.cc:1905] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:38.955931 15167 catalog_manager.cc:4841] TS a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053): tablet 2f1920f0b0a14ebab55db94bc1843067 (table test-table [id=7fd02cc4596f4e628522b63fb14ff0f3]) successfully deleted
W20260810 06:38:38.956849 15165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:42841 (error 108)
I20260810 06:38:38.957080 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 15401
I20260810 06:38:38.957118 15743 ts_tablet_manager.cc:1918] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:38.957187 15743 log.cc:1196] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/2f1920f0b0a14ebab55db94bc1843067
I20260810 06:38:38.957417 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.957280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42841 (local address 127.0.0.1:52098)
0810 06:38:38.957341 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.957381 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:38:38.957479 15743 ts_tablet_manager.cc:1939] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Deleting consensus metadata
W20260810 06:38:38.957528 15165 catalog_manager.cc:4584] TS 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1:42841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f1920f0b0a14ebab55db94bc1843067: Network error: Client connection negotiation failed: client connection to 127.30.71.1:42841: connect: Connection refused (error 111)
I20260810 06:38:38.964576 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 15550
I20260810 06:38:38.964607 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:38.964412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43449 (local address 127.0.0.1:41008)
0810 06:38:38.964515 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:38.964555 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.2:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 06:38:38.972185 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 15148
I20260810 06:38:58.980057 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43125
--webserver_interface=127.30.71.62
--webserver_port=43977
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:38:59.066864 15748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:59.067134 15748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:59.067158 15748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:59.068389 15748 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:38:59.068431 15748 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:38:59.068445 15748 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:38:59.068457 15748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:38:59.068470 15748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:59.068485 15748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:59.068496 15748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:38:59.068509 15748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:38:59.069984 15748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=43977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:59.070192 15748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:59.070446 15748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:59.080646 15757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.080655 15755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.080654 15754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:59.080731 15748 server_base.cc:1034] running on GCE node
I20260810 06:38:59.081254 15748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:59.081614 15748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:59.082784 15748 hybrid_clock.cc:648] HybridClock initialized: now 1786343939082771 us; error 53 us; skew 500 ppm
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:59.085695 15748 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:38:59.086268 15748 webserver.cc:460] Webserver started at http://127.30.71.62:43977/ using document root <none> and password file <none>
I20260810 06:38:59.086535 15748 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:59.086622 15748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:59.088179 15748 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.088900 15766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:59.089089 15748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.089203 15748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "cfac2f610f094886a954296e2b7186cf"
format_stamp: "Formatted at 2026-08-10 06:38:37 on dist-test-slave-w206"
server_key: "79a9a4ddba25ba3774fba63c281838e4"
server_key_iv: "f1196a74415bed1b28b3d54c46a91bde"
server_key_version: "encryptionkey@0"
I20260810 06:38:59.089639 15748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:59.098711 15748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:59.101310 15748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:59.101523 15748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:59.106043 15748 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:43125
I20260810 06:38:59.106077 15822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:43125 every 8 connection(s)
I20260810 06:38:59.106505 15748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:38:59.108280 15824 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:38:59.109036 15824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Bootstrap starting.
I20260810 06:38:59.116631 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 15748
I20260810 06:38:59.117072 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:42841
--local_ip_for_outbound_sockets=127.30.71.1
--tserver_master_addrs=127.30.71.62:43125
--webserver_port=38561
--webserver_interface=127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:59.118664 15824 log.cc:824] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:59.120813 15824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:59.121227 15824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Bootstrap complete.
I20260810 06:38:59.123791 15824 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:59.124032 15824 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfac2f610f094886a954296e2b7186cf, State: Initialized, Role: FOLLOWER
I20260810 06:38:59.124210 15824 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:59.124281 15824 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:38:59.124347 15824 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:38:59.124401 15824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:38:59.125597 15824 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:59.125694 15824 leader_election.cc:302] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfac2f610f094886a954296e2b7186cf; no voters:
I20260810 06:38:59.125919 15824 leader_election.cc:288] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:38:59.125962 15828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:38:59.126086 15828 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 LEADER]: Becoming Leader. State: Replica: cfac2f610f094886a954296e2b7186cf, State: Running, Role: LEADER
I20260810 06:38:59.126222 15824 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:38:59.126245 15828 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:38:59.126681 15829 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cfac2f610f094886a954296e2b7186cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } } }
I20260810 06:38:59.126768 15829 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:59.127123 15830 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfac2f610f094886a954296e2b7186cf. Latest consensus state: current_term: 2 leader_uuid: "cfac2f610f094886a954296e2b7186cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } } }
I20260810 06:38:59.127220 15830 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: This master's current role is: LEADER
I20260810 06:38:59.127406 15834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:38:59.128387 15834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:38:59.128500 15834 catalog_manager.cc:1259] Loaded cluster ID: 5edd772b1e724df89376537a4914969f
I20260810 06:38:59.128527 15834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:38:59.129750 15834 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:38:59.129961 15834 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Loaded TSK: 0
I20260810 06:38:59.148540 15834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Generated new TSK 1
I20260810 06:38:59.148677 15834 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:38:59.206718 15826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:59.207036 15826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:59.207104 15826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:59.208360 15826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:59.208410 15826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:59.208438 15826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:38:59.210031 15826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:42841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=38561
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:59.210296 15826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:59.210570 15826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:59.216113 15858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.216099 15855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.216099 15856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:59.216921 15826 server_base.cc:1034] running on GCE node
I20260810 06:38:59.217167 15826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:59.217455 15826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:59.218618 15826 hybrid_clock.cc:648] HybridClock initialized: now 1786343939218598 us; error 39 us; skew 500 ppm
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:59.221633 15826 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:38:59.222255 15826 webserver.cc:460] Webserver started at http://127.30.71.1:38561/ using document root <none> and password file <none>
I20260810 06:38:59.222545 15826 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:59.222632 15826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:59.224581 15826 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.225600 15866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:59.225883 15826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.225988 15826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "51ad1c6a7c114c138361eaae7cdffbdc"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "69c89089cf632201256f29b7e442fe66"
server_key_iv: "90997936ee013091044eeafe030f38a6"
server_key_version: "encryptionkey@0"
I20260810 06:38:59.226629 15826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:59.268715 15826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:59.271389 15826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:59.271543 15826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:59.271790 15826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:59.272271 15873 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:38:59.273984 15826 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:38:59.274065 15826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 06:38:59.274111 15826 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:38:59.275059 15826 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:38:59.275138 15826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.275175 15873 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap starting.
I20260810 06:38:59.282532 15826 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:42841
I20260810 06:38:59.282593 15988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:42841 every 8 connection(s)
I20260810 06:38:59.283066 15826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:38:59.283951 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 15826
I20260810 06:38:59.284307 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:43449
--local_ip_for_outbound_sockets=127.30.71.2
--tserver_master_addrs=127.30.71.62:43125
--webserver_port=33003
--webserver_interface=127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:59.285449 15873 log.cc:824] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:59.288012 15873 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:38:59.288324 15873 tablet_bootstrap.cc:492] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap complete.
I20260810 06:38:59.289289 15873 ts_tablet_manager.cc:1397] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:59.291147 15873 raft_consensus.cc:348] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:59.291554 15873 raft_consensus.cc:729] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad1c6a7c114c138361eaae7cdffbdc, State: Initialized, Role: FOLLOWER
I20260810 06:38:59.291756 15873 consensus_queue.cc:260] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } }
I20260810 06:38:59.292111 15873 ts_tablet_manager.cc:1428] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20260810 06:38:59.295555 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.283852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:48185)
0810 06:38:59.284277 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.284359 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.285114 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.285909 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.285913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.286222 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:59.286411 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.286417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.287672 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.287673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.288605 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.288608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.289203 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.289205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.289352 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.289966 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:59.289974 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:59.291501 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:59.293750 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.293758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.293760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.294017 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.294302 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.294303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.294304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.294422 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.294817 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:59.294825 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:59.295013 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.295152 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.295345 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":114,"threads_started":1}
I20260810 06:38:59.295562 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.283979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:48185 (local address 127.30.71.62:43125)
0810 06:38:59.285363 (+ 1384us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.285372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.285415 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:38:59.285557 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.285563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.285628 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.285764 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:59.286544 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.287546 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.288693 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.289098 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.289215 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.291708 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:59.291731 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:59.291731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:59.291745 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:59.293658 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.294110 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.294114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.294114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.294221 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.294492 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.294493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.294494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.294737 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:59.294839 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.295212 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.295404 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1322,"thread_start_us":102,"threads_started":1}
I20260810 06:38:59.296250 15989 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43125
I20260810 06:38:59.296391 15989 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:59.296597 15989 heartbeater.cc:511] Master 127.30.71.62:43125 requested a full tablet report, sending...
I20260810 06:38:59.297343 15784 ts_manager.cc:194] Registered new tserver with Master: 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1:42841)
I20260810 06:38:59.298395 15784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:48185
I20260810 06:38:59.299470 15989 heartbeater.cc:503] Master 127.30.71.62:43125 was elected leader, sending a full tablet report...
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.1@KRBTEST.COM
I20260810 06:38:59.308254 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.298071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42841 (local address 127.0.0.1:42988)
0810 06:38:59.298443 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.298486 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.298571 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.298883 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.298889 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.299261 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:59.299327 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.299329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.300517 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.300518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.301475 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.301478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.301944 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.301945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.302032 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.302629 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:59.302635 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:59.304365 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:59.306674 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.306685 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.306687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.306946 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.307341 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.307342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.307343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.307425 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.307781 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:59.307790 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:59.307859 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.307967 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.308140 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":122,"threads_started":1}
I20260810 06:38:59.308254 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.298147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42988 (local address 127.30.71.1:42841)
0810 06:38:59.298365 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.298370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.298665 (+ 295us) server_negotiation.cc:407] Connection header received
0810 06:38:59.298705 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.298707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.298724 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.298792 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:59.299427 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.300431 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.301558 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.301862 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.301923 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.304499 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:59.304517 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:59.304517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:59.304525 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:59.306455 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.307106 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.307110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.307110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.307249 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.307509 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.307509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.307510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.307690 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:59.307806 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.308024 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.308140 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
I20260810 06:38:59.308801 15916 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f1920f0b0a14ebab55db94bc1843067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:38 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:42988
I20260810 06:38:59.309373 15998 tablet_replica.cc:330] stopping tablet replica
I20260810 06:38:59.309523 15998 raft_consensus.cc:2229] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:59.309613 15998 raft_consensus.cc:2258] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:59.309969 15998 ts_tablet_manager.cc:1905] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:59.312366 15998 ts_tablet_manager.cc:1918] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:59.312463 15998 log.cc:1196] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal/wals/2f1920f0b0a14ebab55db94bc1843067
I20260810 06:38:59.312716 15998 ts_tablet_manager.cc:1939] T 2f1920f0b0a14ebab55db94bc1843067 P 51ad1c6a7c114c138361eaae7cdffbdc: Deleting consensus metadata
I20260810 06:38:59.313333 15768 catalog_manager.cc:4841] TS 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1:42841): tablet 2f1920f0b0a14ebab55db94bc1843067 (table test-table [id=7fd02cc4596f4e628522b63fb14ff0f3]) successfully deleted
W20260810 06:38:59.378892 15993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:59.379283 15993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:59.379331 15993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:59.380677 15993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:59.380759 15993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:59.380812 15993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:38:59.382526 15993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:43449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=33003
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:59.382836 15993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:59.383167 15993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:59.388574 16004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.388592 16007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.388592 16005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:59.388741 15993 server_base.cc:1034] running on GCE node
I20260810 06:38:59.389077 15993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:59.389350 15993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:59.390514 15993 hybrid_clock.cc:648] HybridClock initialized: now 1786343939390517 us; error 47 us; skew 500 ppm
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:59.393429 15993 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:38:59.393929 15993 webserver.cc:460] Webserver started at http://127.30.71.2:33003/ using document root <none> and password file <none>
I20260810 06:38:59.394133 15993 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:59.394196 15993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:59.395831 15993 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:38:59.396502 16014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:59.396718 15993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:38:59.396827 15993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "074b579872b842a385332ca1032ffc26"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "41084789244c9c8e749398cd074b562b"
server_key_iv: "1ea0865dcafef6720ec62463c8fcbfe4"
server_key_version: "encryptionkey@0"
I20260810 06:38:59.397223 15993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:59.419085 15993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:59.421499 15993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:59.421675 15993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:59.421957 15993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:59.422451 16021 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260810 06:38:59.424126 16021 ts_tablet_manager.cc:1852] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 06:38:59.424412 16021 ts_tablet_manager.cc:1905] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:59.426903 16021 ts_tablet_manager.cc:1918] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:59.427105 16021 ts_tablet_manager.cc:1939] T 2f1920f0b0a14ebab55db94bc1843067 P 074b579872b842a385332ca1032ffc26: Deleting consensus metadata
I20260810 06:38:59.427453 15993 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 06:38:59.427512 15993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 06:38:59.427588 15993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:59.427620 15993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:59.434370 15993 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:43449
I20260810 06:38:59.434495 16137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:43449 every 8 connection(s)
I20260810 06:38:59.434805 15993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:59.443890 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 15993
I20260810 06:38:59.444281 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:39053
--local_ip_for_outbound_sockets=127.30.71.3
--tserver_master_addrs=127.30.71.62:43125
--webserver_port=41127
--webserver_interface=127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:38:59.444737 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.435679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:44807 (local address 127.30.71.62:43125)
0810 06:38:59.435758 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.435760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.435960 (+ 200us) server_negotiation.cc:407] Connection header received
0810 06:38:59.436502 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.436503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.436517 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.436574 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:59.437309 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.437823 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.438830 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.439105 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.439160 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.441439 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:59.441441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:59.441441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:59.441448 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:59.443016 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.443436 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.443438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.443438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.443503 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.443801 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.443801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.443802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.443924 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:59.444035 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.444492 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.444595 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:38:59.445353 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.435616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:44807)
0810 06:38:59.435827 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.435867 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.436407 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.436637 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.436640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.436994 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:59.437202 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.437207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.437912 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.437913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.438750 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.438752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.439179 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.439180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.439291 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.439868 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:59.439876 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:59.441347 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:59.443099 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.443105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.443107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.443344 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.443574 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.443575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.443575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.443727 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.444060 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:59.444068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:59.444250 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.444853 (+ 603us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.445051 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":114,"threads_started":1}
I20260810 06:38:59.446041 16138 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43125
I20260810 06:38:59.446156 16138 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:59.446411 16138 heartbeater.cc:511] Master 127.30.71.62:43125 requested a full tablet report, sending...
I20260810 06:38:59.446887 15784 ts_manager.cc:194] Registered new tserver with Master: 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449)
I20260810 06:38:59.447527 15784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:44807
W20260810 06:38:59.536880 16142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:38:59.537108 16142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:38:59.537134 16142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:38:59.538273 16142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:38:59.538313 16142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:38:59.538340 16142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:38:59.539858 16142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:39053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=41127
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:38:59.540091 16142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:38:59.540294 16142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:38:59.545648 16150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.545648 16147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:38:59.545691 16148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:38:59.545975 16142 server_base.cc:1034] running on GCE node
I20260810 06:38:59.546223 16142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:38:59.546532 16142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:38:59.547708 16142 hybrid_clock.cc:648] HybridClock initialized: now 1786343939547693 us; error 27 us; skew 500 ppm
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:38:59.550420 16142 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:38:59.550916 16142 webserver.cc:460] Webserver started at http://127.30.71.3:41127/ using document root <none> and password file <none>
I20260810 06:38:59.551179 16142 fs_manager.cc:359] Metadata directory not provided
I20260810 06:38:59.551270 16142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:38:59.552724 16142 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.553464 16157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:59.553771 16142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.553892 16142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "a66ea2fb841b4011a0eaa27be6e8f920"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "9c2c4fa7ec931b8696d6b718af6cfcf4"
server_key_iv: "23c1d5de90f6d9d8a57211919a37ee01"
server_key_version: "encryptionkey@0"
I20260810 06:38:59.554270 16142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:38:59.565042 16142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:38:59.567657 16142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:38:59.567778 16142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:38:59.568075 16142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:38:59.568435 16142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:38:59.568470 16142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:59.568495 16142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:38:59.568506 16142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:38:59.574623 16142 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:39053
I20260810 06:38:59.574712 16279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:39053 every 8 connection(s)
I20260810 06:38:59.575166 16142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:38:59.581465 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 16142
Aug 10 06:38:59 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:38:59.586174 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.576005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:54857 (local address 127.30.71.62:43125)
0810 06:38:59.576054 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.576057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.576295 (+ 238us) server_negotiation.cc:407] Connection header received
0810 06:38:59.576826 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.576827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.576838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.576903 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:59.577981 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.578506 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.579632 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.579822 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.579877 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.582705 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:59.582706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:59.582706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:59.582712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:59.584491 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.585056 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.585057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.585058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.585128 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.585399 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.585400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.585401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.585560 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:59.585635 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.586033 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.586105 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:38:59.586503 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.575959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:54857)
0810 06:38:59.576146 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.576190 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.576729 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.576985 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.576989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.577631 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:38:59.577848 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.577852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.578607 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.578608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.579546 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.579550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.579916 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.579917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.580030 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.580627 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:38:59.580636 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:38:59.582603 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:38:59.584589 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.584597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.584599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.584948 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.585206 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:38:59.585206 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:38:59.585207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:38:59.585324 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:38:59.585640 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:38:59.585647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:38:59.585871 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.586007 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.586201 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260810 06:38:59.587131 16280 heartbeater.cc:347] Connected to a master server at 127.30.71.62:43125
I20260810 06:38:59.587265 16280 heartbeater.cc:464] Registering TS with master...
I20260810 06:38:59.587493 16280 heartbeater.cc:511] Master 127.30.71.62:43125 requested a full tablet report, sending...
I20260810 06:38:59.588032 15784 ts_manager.cc:194] Registered new tserver with Master: a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
I20260810 06:38:59.588570 15784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:54857
I20260810 06:38:59.589834 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.581975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52462 (local address 127.30.71.62:43125)
0810 06:38:59.582112 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.582114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.582244 (+ 130us) server_negotiation.cc:407] Connection header received
0810 06:38:59.582319 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.582320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.582329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.582412 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:59.583227 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.583934 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.584982 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.585189 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.585278 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.586714 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:59.586715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:59.586716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:59.586724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:59.588458 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.589028 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.589029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.589030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.589099 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.589363 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.589364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.589364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.589497 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:59.589664 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.589696 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.589768 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260810 06:38:59.590687 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:38:59.594038 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.591085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.30.71.62:43125)
0810 06:38:59.591151 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.591154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.591251 (+ 97us) server_negotiation.cc:407] Connection header received
0810 06:38:59.591334 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.591335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.591386 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.591480 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:59.591610 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.592155 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.593101 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.593340 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.593443 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.593644 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:59.593708 (+ 64us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 06:38:59.593918 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52476: token expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 06:38:59.594117 16284 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52476: token expired
W20260810 06:38:59.594364 15704 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.71.62:43125)
I20260810 06:38:59.601635 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.594708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.30.71.62:43125)
0810 06:38:59.594747 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.594749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.594819 (+ 70us) server_negotiation.cc:407] Connection header received
0810 06:38:59.594895 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.594895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.594906 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.594968 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:38:59.595543 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.596050 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.596844 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.597068 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.597153 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.598535 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:38:59.598537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:38:59.598537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:38:59.598545 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:38:59.600210 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.600691 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.600693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.600693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.600763 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:38:59.601027 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:38:59.601028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:38:59.601029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:38:59.601155 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:38:59.601266 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.601430 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.601523 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:38:59.603096 15784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:38:59.604866 15784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:38:59.607331 15916 tablet_service.cc:1467] Processing CreateTablet for tablet 94ed434ac73647ae9b67198d86b4567e (DEFAULT_TABLE table=test-table [id=7ccf673cbee7475e9aef703bf62625b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:59.607577 15916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ed434ac73647ae9b67198d86b4567e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:59.609381 15873 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap starting.
I20260810 06:38:59.609756 15873 tablet_bootstrap.cc:654] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:59.610345 15873 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: No bootstrap required, opened a new log
I20260810 06:38:59.610423 15873 ts_tablet_manager.cc:1397] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.610845 15873 raft_consensus.cc:348] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.610931 15873 raft_consensus.cc:374] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:59.611009 15873 raft_consensus.cc:729] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad1c6a7c114c138361eaae7cdffbdc, State: Initialized, Role: FOLLOWER
I20260810 06:38:59.611061 15873 consensus_queue.cc:260] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.611155 15873 ts_tablet_manager.cc:1428] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 06:38:59.611349 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.606932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43449 (local address 127.0.0.1:49376)
0810 06:38:59.607003 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.607027 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.607074 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.607702 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.607703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.607705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:59.607768 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.607770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.608742 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.608743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.609870 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.609873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.611033 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.611034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.611115 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.611141 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.611208 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.611265 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 06:38:59.611366 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.607104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.30.71.2:43449)
0810 06:38:59.607400 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.607406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.607416 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:38:59.607462 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.607464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.607483 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.607718 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:59.607845 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.608635 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.609979 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.610891 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.610990 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.611103 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.611237 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.611280 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":223,"threads_started":1}
I20260810 06:38:59.611999 16064 tablet_service.cc:1467] Processing CreateTablet for tablet 94ed434ac73647ae9b67198d86b4567e (DEFAULT_TABLE table=test-table [id=7ccf673cbee7475e9aef703bf62625b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:59.612241 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.607027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39053 (local address 127.0.0.1:59804)
0810 06:38:59.607166 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.607177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.607218 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.607824 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.607825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.607828 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:59.607882 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.607884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.609323 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.609325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.610524 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.610527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.611956 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.611957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.612041 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.612045 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.612114 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.612160 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260810 06:38:59.612313 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.607286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.30.71.3:39053)
0810 06:38:59.607606 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.607611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.607623 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:38:59.607665 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.607667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.607685 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.607967 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:59.608065 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.609218 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.610628 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.611514 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.611619 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.611686 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.612144 (+ 458us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.612201 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260810 06:38:59.612401 16064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ed434ac73647ae9b67198d86b4567e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:59.613121 16207 tablet_service.cc:1467] Processing CreateTablet for tablet 94ed434ac73647ae9b67198d86b4567e (DEFAULT_TABLE table=test-table [id=7ccf673cbee7475e9aef703bf62625b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:38:59.613474 16207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ed434ac73647ae9b67198d86b4567e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:38:59.614686 16021 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Bootstrap starting.
I20260810 06:38:59.615769 16021 tablet_bootstrap.cc:654] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:59.616194 16021 log.cc:824] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:59.617148 16021 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: No bootstrap required, opened a new log
I20260810 06:38:59.617220 16021 ts_tablet_manager.cc:1397] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 06:38:59.617691 16291 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Bootstrap starting.
I20260810 06:38:59.618366 16291 tablet_bootstrap.cc:654] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Neither blocks nor log segments found. Creating new log.
I20260810 06:38:59.618891 16291 log.cc:824] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Log is configured to *not* fsync() on all Append() calls
I20260810 06:38:59.619105 16021 raft_consensus.cc:348] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.619279 16021 raft_consensus.cc:374] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:59.619509 16021 raft_consensus.cc:729] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074b579872b842a385332ca1032ffc26, State: Initialized, Role: FOLLOWER
I20260810 06:38:59.619733 16021 consensus_queue.cc:260] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.619817 16291 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: No bootstrap required, opened a new log
I20260810 06:38:59.619921 16291 ts_tablet_manager.cc:1397] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:38:59.620121 16021 ts_tablet_manager.cc:1428] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 06:38:59.620126 16138 heartbeater.cc:503] Master 127.30.71.62:43125 was elected leader, sending a full tablet report...
I20260810 06:38:59.621796 16291 raft_consensus.cc:348] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.621969 16291 raft_consensus.cc:374] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:38:59.622017 16291 raft_consensus.cc:729] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66ea2fb841b4011a0eaa27be6e8f920, State: Initialized, Role: FOLLOWER
I20260810 06:38:59.622138 16291 consensus_queue.cc:260] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.622438 16291 ts_tablet_manager.cc:1428] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:38:59.622503 16280 heartbeater.cc:503] Master 127.30.71.62:43125 was elected leader, sending a full tablet report...
W20260810 06:38:59.685504 16139 tablet.cc:2369] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:38:59.692265 16294 raft_consensus.cc:482] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:38:59.692430 16294 raft_consensus.cc:504] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.692878 16294 leader_election.cc:288] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1:42841)
I20260810 06:38:59.696897 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.693238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:47211 (local address 127.30.71.2:43449)
0810 06:38:59.693280 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.693282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.693292 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:38:59.693396 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.693397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.693409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.693463 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:59.693631 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.694397 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.695945 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.696476 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.696555 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.696568 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.696774 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.696829 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:38:59.696895 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.693119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43449 (local address 127.30.71.3:47211)
0810 06:38:59.693193 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.693207 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.693274 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.693467 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.693468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.693469 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:59.693556 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.693558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.694486 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.694487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.695842 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.695846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.696602 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.696604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.696679 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.696683 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.696739 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.696774 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:38:59.697120 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.693133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42841 (local address 127.30.71.3:35859)
0810 06:38:59.693596 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.693615 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:38:59.693699 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:38:59.693867 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:38:59.693867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:38:59.693869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:38:59.693931 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.693933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.694751 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.694752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.696034 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:38:59.696037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.696844 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.696845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:38:59.696917 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.696922 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:38:59.696981 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:38:59.697025 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":274,"thread_start_us":132,"threads_started":1}
I20260810 06:38:59.697261 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.693380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:35859 (local address 127.30.71.1:42841)
0810 06:38:59.693438 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.693441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.693678 (+ 237us) server_negotiation.cc:407] Connection header received
0810 06:38:59.693751 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.693753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.693783 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.693852 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:38:59.694008 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.694663 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.696128 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.696741 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.696813 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.696901 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.697142 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.697193 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:38:59.697350 16088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ed434ac73647ae9b67198d86b4567e" candidate_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074b579872b842a385332ca1032ffc26" is_pre_election: true
I20260810 06:38:59.697584 16088 raft_consensus.cc:2454] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66ea2fb841b4011a0eaa27be6e8f920 in term 0.
I20260810 06:38:59.697595 15938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ed434ac73647ae9b67198d86b4567e" candidate_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" is_pre_election: true
I20260810 06:38:59.697729 15938 raft_consensus.cc:2454] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a66ea2fb841b4011a0eaa27be6e8f920 in term 0.
I20260810 06:38:59.697858 16160 leader_election.cc:302] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074b579872b842a385332ca1032ffc26, a66ea2fb841b4011a0eaa27be6e8f920; no voters:
I20260810 06:38:59.698002 16294 raft_consensus.cc:2781] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:38:59.698060 16294 raft_consensus.cc:482] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:38:59.698096 16294 raft_consensus.cc:3037] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:59.698843 16294 raft_consensus.cc:504] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.699028 16294 leader_election.cc:288] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [CANDIDATE]: Term 1 election: Requested vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1:42841)
I20260810 06:38:59.699195 15938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ed434ac73647ae9b67198d86b4567e" candidate_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51ad1c6a7c114c138361eaae7cdffbdc"
I20260810 06:38:59.699201 16088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ed434ac73647ae9b67198d86b4567e" candidate_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074b579872b842a385332ca1032ffc26"
I20260810 06:38:59.699306 15938 raft_consensus.cc:3037] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:59.699303 16088 raft_consensus.cc:3037] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:38:59.700187 15938 raft_consensus.cc:2454] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66ea2fb841b4011a0eaa27be6e8f920 in term 1.
I20260810 06:38:59.700188 16088 raft_consensus.cc:2454] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a66ea2fb841b4011a0eaa27be6e8f920 in term 1.
I20260810 06:38:59.700372 16159 leader_election.cc:302] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ad1c6a7c114c138361eaae7cdffbdc, a66ea2fb841b4011a0eaa27be6e8f920; no voters:
I20260810 06:38:59.700466 16294 raft_consensus.cc:2781] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:38:59.700644 16294 raft_consensus.cc:686] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 LEADER]: Becoming Leader. State: Replica: a66ea2fb841b4011a0eaa27be6e8f920, State: Running, Role: LEADER
I20260810 06:38:59.700800 16294 consensus_queue.cc:237] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:38:59.701781 15783 catalog_manager.cc:5495] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 reported cstate change: term changed from 0 to 1, leader changed from <none> to a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3). New cstate: current_term: 1 leader_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } health_report { overall_health: UNKNOWN } } }
I20260810 06:38:59.708942 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.706485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.30.71.3:39053)
0810 06:38:59.706552 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:38:59.706554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:38:59.706563 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:38:59.706607 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:38:59.706608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:38:59.706626 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:38:59.706686 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:38:59.706815 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.707384 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.708095 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:38:59.708282 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:38:59.708353 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:38:59.708495 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:38:59.708596 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:38:59.708663 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:38:59.708800 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:38:59.708821 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:38:59.711143 15938 raft_consensus.cc:1261] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Refusing update from remote peer a66ea2fb841b4011a0eaa27be6e8f920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:59.711149 16088 raft_consensus.cc:1261] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Refusing update from remote peer a66ea2fb841b4011a0eaa27be6e8f920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:38:59.711643 16294 consensus_queue.cc:1035] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:59.711848 16296 consensus_queue.cc:1035] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:38:59.714247 16299 mvcc.cc:205] Tried to move back new op lower bound from 7316864777053417472 to 7316864777015267328. Current Snapshot: MvccSnapshot[applied={T|T < 7316864777053417472}]
I20260810 06:38:59.714397 16300 mvcc.cc:205] Tried to move back new op lower bound from 7316864777053417472 to 7316864777015267328. Current Snapshot: MvccSnapshot[applied={T|T < 7316864777053417472}]
I20260810 06:38:59.715323 16301 mvcc.cc:205] Tried to move back new op lower bound from 7316864777053417472 to 7316864777015267328. Current Snapshot: MvccSnapshot[applied={T|T < 7316864777053417472}]
I20260810 06:38:59.717010 15784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:59.717127 15784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:38:59.718012 15784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Sending DeleteTablet for 3 replicas of tablet 94ed434ac73647ae9b67198d86b4567e
I20260810 06:38:59.718374 15916 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ed434ac73647ae9b67198d86b4567e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:59 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:42988
I20260810 06:38:59.718405 16064 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ed434ac73647ae9b67198d86b4567e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:59 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:49376
I20260810 06:38:59.718475 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 15826
I20260810 06:38:59.718557 16207 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ed434ac73647ae9b67198d86b4567e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:38:59 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:59804
I20260810 06:38:59.719108 16310 tablet_replica.cc:330] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: stopping tablet replica
I20260810 06:38:59.719296 16310 raft_consensus.cc:2229] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:38:59.719489 16310 raft_consensus.cc:2258] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:38:59.720134 16310 ts_tablet_manager.cc:1905] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:59.722158 16310 ts_tablet_manager.cc:1918] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:59.722261 16310 log.cc:1196] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/94ed434ac73647ae9b67198d86b4567e
I20260810 06:38:59.722635 16310 ts_tablet_manager.cc:1939] T 94ed434ac73647ae9b67198d86b4567e P 074b579872b842a385332ca1032ffc26: Deleting consensus metadata
I20260810 06:38:59.723284 15770 catalog_manager.cc:4841] TS 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449): tablet 94ed434ac73647ae9b67198d86b4567e (table test-table [id=7ccf673cbee7475e9aef703bf62625b9]) successfully deleted
I20260810 06:38:59.727232 16311 tablet_replica.cc:330] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: stopping tablet replica
I20260810 06:38:59.727408 16311 raft_consensus.cc:2229] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:38:59.727551 16311 raft_consensus.cc:2258] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 06:38:59.727632 15768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:42841 (error 108)
I20260810 06:38:59.727973 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 15993
I20260810 06:38:59.729679 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:38:59.729526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42841 (local address 127.0.0.1:42996)
0810 06:38:59.729591 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:38:59.729637 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:38:59.730075 15768 catalog_manager.cc:4584] TS 51ad1c6a7c114c138361eaae7cdffbdc (127.30.71.1:42841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94ed434ac73647ae9b67198d86b4567e: Network error: Client connection negotiation failed: client connection to 127.30.71.1:42841: connect: Connection refused (error 111)
I20260810 06:38:59.731078 16311 ts_tablet_manager.cc:1905] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:38:59.733095 16311 ts_tablet_manager.cc:1918] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:38:59.733194 16311 log.cc:1196] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/94ed434ac73647ae9b67198d86b4567e
I20260810 06:38:59.733580 16311 ts_tablet_manager.cc:1939] T 94ed434ac73647ae9b67198d86b4567e P a66ea2fb841b4011a0eaa27be6e8f920: Deleting consensus metadata
I20260810 06:38:59.734117 15771 catalog_manager.cc:4841] TS a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053): tablet 94ed434ac73647ae9b67198d86b4567e (table test-table [id=7ccf673cbee7475e9aef703bf62625b9]) successfully deleted
I20260810 06:38:59.736236 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 16142
I20260810 06:38:59.744753 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 15748
I20260810 06:39:19.752219 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43125
--webserver_interface=127.30.71.62
--webserver_port=43977
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:19.844069 16313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:19.844301 16313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:19.844327 16313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:19.845546 16313 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:39:19.845594 16313 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:39:19.845608 16313 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:39:19.845619 16313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:19.845633 16313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:19.845686 16313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:19.845710 16313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:19.845737 16313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:19.847465 16313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=43977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:19.847751 16313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:19.848019 16313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:19.853456 16318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:19.853487 16313 server_base.cc:1034] running on GCE node
W20260810 06:39:19.853507 16322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:19.853487 16320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:19.853967 16313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:19.854344 16313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:19.855520 16313 hybrid_clock.cc:648] HybridClock initialized: now 1786343959855508 us; error 26 us; skew 500 ppm
Aug 10 06:39:19 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343959, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:19.858270 16313 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:39:19.858704 16313 webserver.cc:460] Webserver started at http://127.30.71.62:43977/ using document root <none> and password file <none>
I20260810 06:39:19.859021 16313 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:19.859084 16313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:19.860540 16313 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:19.861495 16329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:19.861801 16313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:19.861944 16313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "cfac2f610f094886a954296e2b7186cf"
format_stamp: "Formatted at 2026-08-10 06:38:37 on dist-test-slave-w206"
server_key: "79a9a4ddba25ba3774fba63c281838e4"
server_key_iv: "f1196a74415bed1b28b3d54c46a91bde"
server_key_version: "encryptionkey@0"
I20260810 06:39:19.862444 16313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:19.876685 16313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:19.879145 16313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:19.879383 16313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:19.883834 16313 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:43125
I20260810 06:39:19.883877 16391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:43125 every 8 connection(s)
I20260810 06:39:19.884316 16313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:19.885993 16392 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:39:19.886786 16392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Bootstrap starting.
I20260810 06:39:19.889397 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 16313
I20260810 06:39:19.889757 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:42841
--local_ip_for_outbound_sockets=127.30.71.1
--tserver_master_addrs=127.30.71.62:43125
--webserver_port=38561
--webserver_interface=127.30.71.1
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:39:19.896443 16392 log.cc:824] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:19.898833 16392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:39:19.899248 16392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Bootstrap complete.
I20260810 06:39:19.901755 16392 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:39:19.902048 16392 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfac2f610f094886a954296e2b7186cf, State: Initialized, Role: FOLLOWER
I20260810 06:39:19.902242 16392 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:39:19.902324 16392 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:19.902383 16392 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:19.902441 16392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:39:19.903584 16392 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:39:19.903709 16392 leader_election.cc:302] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfac2f610f094886a954296e2b7186cf; no voters:
I20260810 06:39:19.903929 16392 leader_election.cc:288] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:39:19.903970 16396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:39:19.904090 16396 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [term 3 LEADER]: Becoming Leader. State: Replica: cfac2f610f094886a954296e2b7186cf, State: Running, Role: LEADER
I20260810 06:39:19.904238 16392 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:19.904327 16396 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } }
I20260810 06:39:19.904663 16396 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfac2f610f094886a954296e2b7186cf. Latest consensus state: current_term: 3 leader_uuid: "cfac2f610f094886a954296e2b7186cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } } }
I20260810 06:39:19.904778 16396 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:19.904670 16398 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cfac2f610f094886a954296e2b7186cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfac2f610f094886a954296e2b7186cf" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 43125 } } }
I20260810 06:39:19.904980 16398 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:19.905913 16403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:19.906852 16403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:19.907011 16403 catalog_manager.cc:1259] Loaded cluster ID: 5edd772b1e724df89376537a4914969f
I20260810 06:39:19.907048 16403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:19.908334 16403 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:19.908536 16403 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Loaded TSK: 0
I20260810 06:39:19.908583 16403 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfac2f610f094886a954296e2b7186cf: Loaded TSK: 1
F20260810 06:39:19.908841 16403 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe669dd59ad google::LogMessage::Fail() at ??:0
@ 0x7fe669dd888a google::LogMessage::SendToLog() at ??:0
@ 0x7fe669dd54de google::LogMessage::Flush() at ??:0
@ 0x7fe669dd6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe66c542104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe66c53fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe671d2f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe671d2f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe671d2f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe671cf5c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe671cffad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe66bca30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe66bc98954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe66cafc6db start_thread at ??:0
@ 0x7fe6688d361f clone at ??:0
W20260810 06:39:19.999742 16394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:20.000001 16394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:20.000036 16394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:20.001315 16394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:20.001372 16394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:20.001401 16394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:39:20.003226 16394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:42841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=38561
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:20.003517 16394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:20.003788 16394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:20.009361 16438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:20.009346 16435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:20.009338 16436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:20.009680 16394 server_base.cc:1034] running on GCE node
I20260810 06:39:20.009979 16394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:20.010254 16394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:20.011476 16394 hybrid_clock.cc:648] HybridClock initialized: now 1786343960011458 us; error 33 us; skew 500 ppm
Aug 10 06:39:20 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343960, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:20.014271 16394 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:39:20.014817 16394 webserver.cc:460] Webserver started at http://127.30.71.1:38561/ using document root <none> and password file <none>
I20260810 06:39:20.015120 16394 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:20.015194 16394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:20.016731 16394 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:20.017735 16445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:20.017975 16394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:20.018092 16394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "51ad1c6a7c114c138361eaae7cdffbdc"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "69c89089cf632201256f29b7e442fe66"
server_key_iv: "90997936ee013091044eeafe030f38a6"
server_key_version: "encryptionkey@0"
I20260810 06:39:20.018553 16394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:20.030936 16394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:20.033533 16394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:20.033721 16394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:20.033990 16394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:20.034523 16452 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 06:39:20.036710 16394 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 06:39:20.036792 16394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 06:39:20.036840 16394 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 06:39:20.037564 16394 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 06:39:20.037626 16394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 06:39:20.037704 16452 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap starting.
I20260810 06:39:20.045341 16394 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:42841
I20260810 06:39:20.045394 16569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:42841 every 8 connection(s)
I20260810 06:39:20.045814 16394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:39:20.046494 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 16394
I20260810 06:39:20.046810 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:43449
--local_ip_for_outbound_sockets=127.30.71.2
--tserver_master_addrs=127.30.71.62:43125
--webserver_port=33003
--webserver_interface=127.30.71.2
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:39:20.049925 16452 log.cc:824] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:20.052882 16452 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:39:20.053261 16452 tablet_bootstrap.cc:492] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Bootstrap complete.
I20260810 06:39:20.054594 16452 ts_tablet_manager.cc:1397] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.005s
I20260810 06:39:20.056193 16452 raft_consensus.cc:348] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:39:20.056589 16452 raft_consensus.cc:729] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad1c6a7c114c138361eaae7cdffbdc, State: Initialized, Role: FOLLOWER
I20260810 06:39:20.056771 16452 consensus_queue.cc:260] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:39:20.057106 16452 ts_tablet_manager.cc:1428] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260810 06:39:20.125671 16573 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:39:20.048760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:45005)
0810 06:39:20.049004 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.049065 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:20.049682 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:20.125429 (+ 75747us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":125,"threads_started":1}
W20260810 06:39:20.125931 16570 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:43125 (0 consecutive failures): Network error: Failed to ping master at 127.30.71.62:43125: Client connection negotiation failed: client connection to 127.30.71.62:43125: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:39:20.126303 16573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.126171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:51099)
0810 06:39:20.126231 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.126273 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 06:39:20.126729 16573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.126643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:52445)
0810 06:39:20.126691 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.126715 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:39:20.126864 16570 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:39:20.135556 16572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:20.135777 16572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:20.135804 16572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:20.137060 16572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:20.137110 16572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:20.137140 16572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:39:20.138723 16572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:43449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=33003
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:20.139046 16572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:20.139341 16572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:20.144805 16583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:20.144807 16580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:20.144990 16572 server_base.cc:1034] running on GCE node
W20260810 06:39:20.145246 16581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:20.145488 16572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:20.145788 16572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:20.147034 16572 hybrid_clock.cc:648] HybridClock initialized: now 1786343960147032 us; error 66 us; skew 500 ppm
Aug 10 06:39:20 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343960, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:20.149796 16572 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:39:20.150342 16572 webserver.cc:460] Webserver started at http://127.30.71.2:33003/ using document root <none> and password file <none>
I20260810 06:39:20.150624 16572 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:20.150697 16572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:20.152241 16572 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:20.152993 16592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:20.153244 16572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:20.153582 16572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "074b579872b842a385332ca1032ffc26"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "41084789244c9c8e749398cd074b562b"
server_key_iv: "1ea0865dcafef6720ec62463c8fcbfe4"
server_key_version: "encryptionkey@0"
I20260810 06:39:20.153987 16572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:20.172936 16572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:20.175714 16572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:20.175855 16572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:20.176160 16572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:20.176517 16572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:20.176553 16572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:20.176576 16572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:20.176623 16572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:20.183106 16572 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:43449
I20260810 06:39:20.183156 16720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:43449 every 8 connection(s)
I20260810 06:39:20.183557 16572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:39:20.183881 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 16572
I20260810 06:39:20.184235 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:39053
--local_ip_for_outbound_sockets=127.30.71.3
--tserver_master_addrs=127.30.71.62:43125
--webserver_port=41127
--webserver_interface=127.30.71.3
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:39:20.184957 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.184446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:47769)
0810 06:39:20.184644 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.184747 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
W20260810 06:39:20.185431 16721 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:43125 (0 consecutive failures): Network error: Failed to ping master at 127.30.71.62:43125: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
I20260810 06:39:20.185781 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.185648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:60921)
0810 06:39:20.185707 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.185746 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 06:39:20.186131 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.186008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:39869)
0810 06:39:20.186063 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.186098 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 06:39:20.186308 16721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:39:20.276259 16724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:20.276484 16724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:20.276510 16724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:20.277746 16724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:20.277807 16724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:20.277835 16724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:39:20.279488 16724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:39053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=41127
--tserver_master_addrs=127.30.71.62:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:20.279757 16724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:20.280031 16724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:20.285729 16732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:20.285709 16730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:20.285709 16729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:20.286046 16724 server_base.cc:1034] running on GCE node
I20260810 06:39:20.286233 16724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:20.286469 16724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:20.287616 16724 hybrid_clock.cc:648] HybridClock initialized: now 1786343960287605 us; error 35 us; skew 500 ppm
Aug 10 06:39:20 dist-test-slave-w206 krb5kdc[15134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343960, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:20.290266 16724 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:39:20.290756 16724 webserver.cc:460] Webserver started at http://127.30.71.3:41127/ using document root <none> and password file <none>
I20260810 06:39:20.290961 16724 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:20.291079 16724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:20.292524 16724 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:20.293303 16739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:20.293598 16724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:20.293722 16724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "a66ea2fb841b4011a0eaa27be6e8f920"
format_stamp: "Formatted at 2026-08-10 06:38:38 on dist-test-slave-w206"
server_key: "9c2c4fa7ec931b8696d6b718af6cfcf4"
server_key_iv: "23c1d5de90f6d9d8a57211919a37ee01"
server_key_version: "encryptionkey@0"
I20260810 06:39:20.294121 16724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:20.321101 16724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:20.323661 16724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:20.323849 16724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:20.324117 16724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:20.324468 16724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:20.324527 16724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:20.324563 16724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:20.324592 16724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:20.331113 16724 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:39053
I20260810 06:39:20.331157 16863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:39053 every 8 connection(s)
I20260810 06:39:20.331545 16724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:39:20.332708 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.332317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:48661)
0810 06:39:20.332480 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.332556 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
W20260810 06:39:20.333161 16864 heartbeater.cc:644] Failed to heartbeat to 127.30.71.62:43125 (0 consecutive failures): Network error: Failed to ping master at 127.30.71.62:43125: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
I20260810 06:39:20.333465 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.333371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:46141)
0810 06:39:20.333417 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.333446 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 06:39:20.333741 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:20.333666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:36157)
0810 06:39:20.333706 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:20.333727 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 06:39:20.333884 16864 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:39:20.341127 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 16724
I20260810 06:39:21.127589 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:21.127280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:36997)
0810 06:39:21.127464 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:21.127511 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260810 06:39:21.187036 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:21.186709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:36313)
0810 06:39:21.186861 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:21.186907 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260810 06:39:21.334514 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:21.334257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:36969)
0810 06:39:21.334409 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:21.334452 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
I20260810 06:39:21.520210 16871 raft_consensus.cc:482] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:21.520408 16871 raft_consensus.cc:504] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:39:21.520780 16871 leader_election.cc:288] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
Aug 10 06:39:21 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343960, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.2@KRBTEST.COM
Aug 10 06:39:21 dist-test-slave-w206 krb5kdc[15134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343960, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.3@KRBTEST.COM
I20260810 06:39:21.535470 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:21.521065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:32853 (local address 127.30.71.2:43449)
0810 06:39:21.521234 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:21.521246 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:21.521284 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:39:21.521774 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:21.521801 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:21.521845 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:21.521963 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:21.522918 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.523844 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.524917 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.525340 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.525466 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:21.531508 (+ 6042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:21.531513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:21.531513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:21.531526 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:21.533548 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:21.534086 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:21.534089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:21.534089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:21.534190 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:21.534467 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:21.534468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:21.534468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:21.534762 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:21.534829 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:21.535207 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 06:39:21.535348 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260810 06:39:21.535458 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:21.521057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39053 (local address 127.30.71.1:41507)
0810 06:39:21.521199 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:21.521209 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:21.521257 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:21.522074 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:21.522076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:21.522557 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:21.522764 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:21.522771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.523985 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.523985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:21.524819 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:21.524821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.525431 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.525432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:21.525548 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:21.529592 (+ 4044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:21.529602 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:21.531668 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:21.533755 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:21.533756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:21.533757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:21.533991 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:21.534266 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:21.534266 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:21.534267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:21.534392 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:21.534882 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:21.534883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:21.535027 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 06:39:21.535151 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 06:39:21.535359 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":64,"thread_start_us":69,"threads_started":1}
I20260810 06:39:21.535591 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:21.520967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:43449 (local address 127.30.71.1:32853)
0810 06:39:21.521025 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:21.521039 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:21.521105 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:21.522038 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:21.522042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:21.522555 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:21.522765 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:21.522771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.523955 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.523956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:21.524816 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:21.524819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.525447 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.525448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:21.525553 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:21.529609 (+ 4056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:21.529614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:21.531329 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:21.533650 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:21.533659 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:21.533661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:21.533990 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:21.534319 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:21.534319 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:21.534320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:21.534396 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:21.534841 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:21.534849 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:21.535031 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 06:39:21.535151 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:39:21.535446 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":187}
I20260810 06:39:21.535491 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:21.521198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:41507 (local address 127.30.71.3:39053)
0810 06:39:21.521382 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:21.521390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:21.521422 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:39:21.521852 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:21.521854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:21.521891 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:21.522051 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:21.522918 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.523865 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.524929 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:21.525336 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:21.525454 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:21.531796 (+ 6342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:21.531800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:21.531800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:21.531811 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:21.533662 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:21.534085 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:21.534089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:21.534089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:21.534189 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:21.534514 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:21.534514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:21.534515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:21.534780 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:21.534888 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:21.535209 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 06:39:21.535348 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
W20260810 06:39:21.536430 16448 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
W20260810 06:39:21.536526 16449 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
I20260810 06:39:21.536600 16449 leader_election.cc:302] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51ad1c6a7c114c138361eaae7cdffbdc; no voters: 074b579872b842a385332ca1032ffc26, a66ea2fb841b4011a0eaa27be6e8f920
I20260810 06:39:21.536705 16871 raft_consensus.cc:2726] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:39:22.128403 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:22.128101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:41161)
0810 06:39:22.128310 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:22.128356 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":70,"threads_started":1}
I20260810 06:39:22.187825 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:22.187588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:50973)
0810 06:39:22.187734 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:22.187778 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260810 06:39:22.335376 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:22.335108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:37443)
0810 06:39:22.335256 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:22.335330 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260810 06:39:23.129244 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:23.129015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:44475)
0810 06:39:23.129150 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:23.129195 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260810 06:39:23.188575 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:23.188336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:48895)
0810 06:39:23.188487 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:23.188527 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260810 06:39:23.289764 16880 raft_consensus.cc:482] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:23.289911 16880 raft_consensus.cc:504] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:39:23.290146 16880 leader_election.cc:288] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
W20260810 06:39:23.290725 16448 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
W20260810 06:39:23.290807 16449 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
I20260810 06:39:23.290846 16449 leader_election.cc:302] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51ad1c6a7c114c138361eaae7cdffbdc; no voters: 074b579872b842a385332ca1032ffc26, a66ea2fb841b4011a0eaa27be6e8f920
I20260810 06:39:23.290927 16880 raft_consensus.cc:2726] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:39:23.336323 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:23.335998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:34907)
0810 06:39:23.336153 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:23.336200 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260810 06:39:24.130046 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:24.129806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:57077)
0810 06:39:24.129953 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:24.129999 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260810 06:39:24.189527 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:24.189174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:50729)
0810 06:39:24.189417 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:24.189470 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":130,"threads_started":1}
I20260810 06:39:24.337126 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:24.336883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:41575)
0810 06:39:24.337024 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:24.337075 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260810 06:39:25.131079 16885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:25.130752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:57395)
0810 06:39:25.130918 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:25.130967 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260810 06:39:25.190361 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:25.190087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:38019)
0810 06:39:25.190254 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:25.190307 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
I20260810 06:39:25.337877 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:25.337663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:41273)
0810 06:39:25.337787 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:25.337826 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260810 06:39:26.131947 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:26.131658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:60137)
0810 06:39:26.131841 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:26.131897 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260810 06:39:26.144613 16889 raft_consensus.cc:482] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:26.144726 16889 raft_consensus.cc:504] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:39:26.144927 16889 leader_election.cc:288] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
W20260810 06:39:26.145399 16448 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
W20260810 06:39:26.145514 16449 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
I20260810 06:39:26.145601 16449 leader_election.cc:302] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51ad1c6a7c114c138361eaae7cdffbdc; no voters: 074b579872b842a385332ca1032ffc26, a66ea2fb841b4011a0eaa27be6e8f920
I20260810 06:39:26.145696 16889 raft_consensus.cc:2726] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:39:26.191303 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:26.191065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:51155)
0810 06:39:26.191196 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:26.191246 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260810 06:39:26.338641 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:26.338396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:43799)
0810 06:39:26.338533 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:26.338579 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260810 06:39:27.132858 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:27.132611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:40971)
0810 06:39:27.132771 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:27.132810 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260810 06:39:27.192068 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:27.191824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:47603)
0810 06:39:27.191976 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:27.192020 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260810 06:39:27.339427 16895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:27.339149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:52937)
0810 06:39:27.339333 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:27.339376 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260810 06:39:28.133832 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:28.133536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:53701)
0810 06:39:28.133721 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:28.133778 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":77,"threads_started":1}
I20260810 06:39:28.192843 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:28.192603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:45295)
0810 06:39:28.192754 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:28.192796 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260810 06:39:28.340159 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:28.339925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:43915)
0810 06:39:28.340068 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:28.340109 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260810 06:39:29.134701 16899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:29.134431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:34715)
0810 06:39:29.134598 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:29.134642 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260810 06:39:29.193692 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:29.193437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:49249)
0810 06:39:29.193596 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:29.193645 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260810 06:39:29.340965 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:29.340706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:60157)
0810 06:39:29.340869 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:29.340916 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260810 06:39:30.135605 16902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:30.135345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:57317)
0810 06:39:30.135514 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:30.135558 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260810 06:39:30.194530 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:30.194238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:60267)
0810 06:39:30.194436 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:30.194481 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":138,"threads_started":1}
I20260810 06:39:30.315519 16904 raft_consensus.cc:482] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:30.315634 16904 raft_consensus.cc:504] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:39:30.315838 16904 leader_election.cc:288] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
W20260810 06:39:30.316358 16449 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
W20260810 06:39:30.316452 16448 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
I20260810 06:39:30.316509 16448 leader_election.cc:302] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51ad1c6a7c114c138361eaae7cdffbdc; no voters: 074b579872b842a385332ca1032ffc26, a66ea2fb841b4011a0eaa27be6e8f920
I20260810 06:39:30.316593 16904 raft_consensus.cc:2726] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:39:30.341683 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:30.341449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:36087)
0810 06:39:30.341585 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:30.341618 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260810 06:39:31.136358 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:31.136108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:42339)
0810 06:39:31.136256 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:31.136301 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260810 06:39:31.195364 16907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:31.195115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:51193)
0810 06:39:31.195262 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:31.195314 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260810 06:39:31.342515 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:31.342240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:60643)
0810 06:39:31.342411 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:31.342455 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
I20260810 06:39:32.137143 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:32.136903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:53903)
0810 06:39:32.137047 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:32.137093 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260810 06:39:32.196101 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:32.195870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:44769)
0810 06:39:32.196022 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:32.196054 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260810 06:39:32.343400 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:32.343138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:53967)
0810 06:39:32.343301 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:32.343346 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":73,"threads_started":1}
I20260810 06:39:32.702337 16912 raft_consensus.cc:482] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:32.702440 16912 raft_consensus.cc:504] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "074b579872b842a385332ca1032ffc26" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 43449 } } peers { permanent_uuid: "a66ea2fb841b4011a0eaa27be6e8f920" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39053 } } peers { permanent_uuid: "51ad1c6a7c114c138361eaae7cdffbdc" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42841 } }
I20260810 06:39:32.702605 16912 leader_election.cc:288] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449), a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053)
W20260810 06:39:32.703049 16448 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 074b579872b842a385332ca1032ffc26 (127.30.71.2:43449): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
W20260810 06:39:32.703158 16449 leader_election.cc:341] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a66ea2fb841b4011a0eaa27be6e8f920 (127.30.71.3:39053): Not found: Tablet not found: 94ed434ac73647ae9b67198d86b4567e
I20260810 06:39:32.703195 16449 leader_election.cc:302] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51ad1c6a7c114c138361eaae7cdffbdc; no voters: 074b579872b842a385332ca1032ffc26, a66ea2fb841b4011a0eaa27be6e8f920
I20260810 06:39:32.703328 16912 raft_consensus.cc:2726] T 94ed434ac73647ae9b67198d86b4567e P 51ad1c6a7c114c138361eaae7cdffbdc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 06:39:33.137933 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:33.137676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:34325)
0810 06:39:33.137840 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:33.137885 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260810 06:39:33.196971 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:33.196718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:60029)
0810 06:39:33.196877 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:33.196923 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260810 06:39:33.344282 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:33.344013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:45051)
0810 06:39:33.344181 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:33.344232 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260810 06:39:34.138731 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:34.138503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:52365)
0810 06:39:34.138635 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:34.138683 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260810 06:39:34.197796 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:34.197461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:45837)
0810 06:39:34.197703 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:34.197747 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":59,"threads_started":1}
I20260810 06:39:34.345027 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:34.344782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:42133)
0810 06:39:34.344940 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:34.344980 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260810 06:39:35.139659 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:35.139394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.1:52155)
0810 06:39:35.139547 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:35.139600 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260810 06:39:35.198629 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:35.198408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.2:41437)
0810 06:39:35.198537 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:35.198581 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260810 06:39:35.345924 16923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:35.345663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:43125 (local address 127.30.71.3:57051)
0810 06:39:35.345824 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:35.345874 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.62:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
I20260810 06:39:35.349265 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 16394
I20260810 06:39:35.355616 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 16572
I20260810 06:39:35.361984 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 16724
2026-08-10T06:39:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58715 ms)
[----------] 30 tests from SecurityITest (241065 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:35 dist-test-slave-w206 krb5kdc[16926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:35 dist-test-slave-w206 krb5kdc[16926](info): set up 2 sockets
Aug 10 06:39:35 dist-test-slave-w206 krb5kdc[16926](info): commencing operation
krb5kdc: starting...
W20260810 06:39:36.483465 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:36 dist-test-slave-w206 krb5kdc[16926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:39:36.616214 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:40725
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:40725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:36.704077 16940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:36.704308 16940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:36.704334 16940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:36.705615 16940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:36.705662 16940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:36.705678 16940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:36.705693 16940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:36.705739 16940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:36.707393 16940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:40725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:40725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:36.707652 16940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:36.707924 16940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:36.713670 16946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:36.713610 16947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:36.713795 16940 server_base.cc:1034] running on GCE node
W20260810 06:39:36.713702 16949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:36.714165 16940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:36.714439 16940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:36.715592 16940 hybrid_clock.cc:648] HybridClock initialized: now 1786343976715567 us; error 36 us; skew 500 ppm
Aug 10 06:39:36 dist-test-slave-w206 krb5kdc[16926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343976, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:36.718381 16940 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:39:36.718847 16940 webserver.cc:460] Webserver started at http://127.30.71.62:42631/ using document root <none> and password file <none>
I20260810 06:39:36.719178 16940 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:36.719257 16940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:36.719419 16940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:36.720554 16940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "50f1d42c62004fab8a8d947dbfcb2f07"
format_stamp: "Formatted at 2026-08-10 06:39:36 on dist-test-slave-w206"
server_key: "622fa12b97080cbdafeb763b81b1c711"
server_key_iv: "cdda88afb7f798e2eb164f39d4eb9681"
server_key_version: "encryptionkey@0"
I20260810 06:39:36.720966 16940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "50f1d42c62004fab8a8d947dbfcb2f07"
format_stamp: "Formatted at 2026-08-10 06:39:36 on dist-test-slave-w206"
server_key: "622fa12b97080cbdafeb763b81b1c711"
server_key_iv: "cdda88afb7f798e2eb164f39d4eb9681"
server_key_version: "encryptionkey@0"
I20260810 06:39:36.722533 16940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:36.723371 16956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:36.723546 16940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:36.723654 16940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "50f1d42c62004fab8a8d947dbfcb2f07"
format_stamp: "Formatted at 2026-08-10 06:39:36 on dist-test-slave-w206"
server_key: "622fa12b97080cbdafeb763b81b1c711"
server_key_iv: "cdda88afb7f798e2eb164f39d4eb9681"
server_key_version: "encryptionkey@0"
I20260810 06:39:36.723731 16940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:36.745779 16940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:36.747879 16940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:36.748059 16940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:36.752193 16940 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:40725
I20260810 06:39:36.752292 17016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:40725 every 8 connection(s)
I20260810 06:39:36.752645 16940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:36.753587 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:36.756029 17017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07: Bootstrap starting.
I20260810 06:39:36.756628 17017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:36.756940 17017 log.cc:824] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:36.757653 17017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07: No bootstrap required, opened a new log
I20260810 06:39:36.759357 17017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40725 } }
I20260810 06:39:36.759496 17017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:36.759519 17017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f1d42c62004fab8a8d947dbfcb2f07, State: Initialized, Role: FOLLOWER
I20260810 06:39:36.759716 17017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40725 } }
I20260810 06:39:36.759805 17017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:36.759857 17017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:36.759922 17017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:36.760471 17017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40725 } }
I20260810 06:39:36.760586 17017 leader_election.cc:302] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50f1d42c62004fab8a8d947dbfcb2f07; no voters:
I20260810 06:39:36.760772 17017 leader_election.cc:288] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:36.760838 17020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:36.761014 17020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [term 1 LEADER]: Becoming Leader. State: Replica: 50f1d42c62004fab8a8d947dbfcb2f07, State: Running, Role: LEADER
I20260810 06:39:36.761035 17017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:36.761144 17020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40725 } }
I20260810 06:39:36.761585 17022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f1d42c62004fab8a8d947dbfcb2f07. Latest consensus state: current_term: 1 leader_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40725 } } }
I20260810 06:39:36.761669 17022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:36.761718 17021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f1d42c62004fab8a8d947dbfcb2f07" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 40725 } } }
I20260810 06:39:36.761778 17021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:36.761905 17029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:36.762151 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 16940
I20260810 06:39:36.762249 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:36.762475 31004 external_mini_cluster.cc:1428] Setting key 48058b01bd22269785c15c11ab9bed3b
I20260810 06:39:36.762652 17029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:36.764462 17029 catalog_manager.cc:1347] Generated new cluster ID: 28073c469cc94a4794a71a3a971b5560
I20260810 06:39:36.764540 17029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:39:36 dist-test-slave-w206 krb5kdc[16926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:36.772773 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:36.762934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.30.71.62:40725)
0810 06:39:36.763467 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:36.763475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:36.763499 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:39:36.763587 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:36.763592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:36.763634 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:36.763773 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:36.764698 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:36.765611 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:36.766405 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:36.766839 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:36.766957 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:36.769414 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:36.769418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:36.769418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:36.769430 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:36.771224 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:36.771592 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:36.771601 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:36.771602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:36.771698 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:36.771913 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:36.771914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:36.771914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:36.772163 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:36.772345 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:36.772423 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 06:39:36.772569 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":95,"threads_started":1}
I20260810 06:39:36.778966 17029 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:36.779716 17029 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:36.783892 17029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07: Generated new TSK 0
I20260810 06:39:36.784075 17029 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:39:36.817734 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:40725
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:36.900698 17044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:36.900919 17044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:36.900942 17044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:36.902105 17044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:36.902150 17044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:36.902204 17044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:39:36.903827 17044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:40725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:36.904085 17044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:36.904340 17044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:36.909482 17049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:36.909525 17052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:36.909510 17050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:36.909870 17044 server_base.cc:1034] running on GCE node
I20260810 06:39:36.910130 17044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:36.910382 17044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:36.911551 17044 hybrid_clock.cc:648] HybridClock initialized: now 1786343976911546 us; error 38 us; skew 500 ppm
Aug 10 06:39:36 dist-test-slave-w206 krb5kdc[16926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343976, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:36.914237 17044 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:39:36.914749 17044 webserver.cc:460] Webserver started at http://127.30.71.1:38651/ using document root <none> and password file <none>
I20260810 06:39:36.915046 17044 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:36.915117 17044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:36.915282 17044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:36.916355 17044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907"
format_stamp: "Formatted at 2026-08-10 06:39:36 on dist-test-slave-w206"
server_key: "6b980c6c6d15ebb9395a73b9d0832172"
server_key_iv: "7365d928486aa233d8b5ce9e1004c5a7"
server_key_version: "encryptionkey@0"
I20260810 06:39:36.916723 17044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907"
format_stamp: "Formatted at 2026-08-10 06:39:36 on dist-test-slave-w206"
server_key: "6b980c6c6d15ebb9395a73b9d0832172"
server_key_iv: "7365d928486aa233d8b5ce9e1004c5a7"
server_key_version: "encryptionkey@0"
I20260810 06:39:36.918053 17044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:36.918866 17059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:36.919090 17044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:36.919201 17044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907"
format_stamp: "Formatted at 2026-08-10 06:39:36 on dist-test-slave-w206"
server_key: "6b980c6c6d15ebb9395a73b9d0832172"
server_key_iv: "7365d928486aa233d8b5ce9e1004c5a7"
server_key_version: "encryptionkey@0"
I20260810 06:39:36.919313 17044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:36.932695 17044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:36.935032 17044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:36.935218 17044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:36.935490 17044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:36.935814 17044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:36.935870 17044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:36.935904 17044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:36.935931 17044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:36.941843 17044 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:44615
I20260810 06:39:36.941887 17180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:44615 every 8 connection(s)
I20260810 06:39:36.942214 17044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:39:36.943203 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 17044
I20260810 06:39:36.943332 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:39:36.943542 31004 external_mini_cluster.cc:1428] Setting key 41b22646473fc19313705993faa90b58
Aug 10 06:39:36 dist-test-slave-w206 krb5kdc[16926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343976, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:39:36.953377 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:36.943023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:37201 (local address 127.30.71.62:40725)
0810 06:39:36.943088 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:36.943091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:36.943370 (+ 279us) server_negotiation.cc:407] Connection header received
0810 06:39:36.944023 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:36.944024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:36.944036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:36.944097 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:36.945129 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:36.945936 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:36.947638 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:36.947849 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:36.947910 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:36.950481 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:36.950483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:36.950483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:36.950492 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:36.951959 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:36.952377 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:36.952378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:36.952379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:36.952441 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:36.952701 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:36.952702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:36.952702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:36.952833 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:36.952910 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:36.953189 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 06:39:36.953283 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:39:36.953580 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:36.942921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:40725 (local address 127.30.71.1:37201)
0810 06:39:36.943237 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:36.943276 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:36.943919 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:36.944167 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:36.944172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:36.944834 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:36.945029 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:36.945033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:36.946147 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:36.946148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:36.947530 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:36.947534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:36.947939 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:36.947941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:36.948091 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:36.948721 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:36.948733 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:36.950376 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:36.952056 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:36.952063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:36.952065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:36.952298 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:36.952515 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:36.952515 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:36.952516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:36.952626 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:36.952900 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:36.952908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:36.953054 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 06:39:36.953166 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:39:36.953338 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":115,"threads_started":1}
I20260810 06:39:36.954187 17181 heartbeater.cc:347] Connected to a master server at 127.30.71.62:40725
I20260810 06:39:36.954324 17181 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:36.954479 17181 heartbeater.cc:511] Master 127.30.71.62:40725 requested a full tablet report, sending...
I20260810 06:39:36.955098 16975 ts_manager.cc:194] Registered new tserver with Master: 9d7ca20bdc2d4ed2a8065e7f50deb907 (127.30.71.1:44615)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
I20260810 06:39:36.955935 16975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:37201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:39:36.970139 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:40725
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:37.052189 17186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:37.052399 17186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:37.052423 17186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:37.053586 17186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:37.053632 17186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:37.053658 17186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:39:37.055186 17186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:40725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:37.055429 17186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:37.055658 17186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:37.060772 17192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.060823 17194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.060829 17191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.060840 17186 server_base.cc:1034] running on GCE node
I20260810 06:39:37.061167 17186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:37.061368 17186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:37.062511 17186 hybrid_clock.cc:648] HybridClock initialized: now 1786343977062490 us; error 37 us; skew 500 ppm
Aug 10 06:39:37 dist-test-slave-w206 krb5kdc[16926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:37.065196 17186 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:39:37.065676 17186 webserver.cc:460] Webserver started at http://127.30.71.2:39607/ using document root <none> and password file <none>
I20260810 06:39:37.065914 17186 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:37.065986 17186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:37.066143 17186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:37.067312 17186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "ffe5c893a4884834b2c27060f05d00bb"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "ac85cfbf6e042e31f759a4b55ccfdab9"
server_key_iv: "6ae22ce23faebcfacda5512661dff246"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.067675 17186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "ffe5c893a4884834b2c27060f05d00bb"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "ac85cfbf6e042e31f759a4b55ccfdab9"
server_key_iv: "6ae22ce23faebcfacda5512661dff246"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.069072 17186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:39:37.069799 17201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.069976 17186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:39:37.070082 17186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "ffe5c893a4884834b2c27060f05d00bb"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "ac85cfbf6e042e31f759a4b55ccfdab9"
server_key_iv: "6ae22ce23faebcfacda5512661dff246"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.070158 17186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:37.102842 17186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:37.105398 17186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:37.105578 17186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:37.105907 17186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:37.106241 17186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:37.106302 17186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.106345 17186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:37.106374 17186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.112716 17186 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:46697
I20260810 06:39:37.112771 17321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:46697 every 8 connection(s)
I20260810 06:39:37.113147 17186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:39:37.115947 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 17186
I20260810 06:39:37.116060 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:39:37.116257 31004 external_mini_cluster.cc:1428] Setting key 86afe595442e041bdd738e9f76e5f093
Aug 10 06:39:37 dist-test-slave-w206 krb5kdc[16926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:37.124079 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.113915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:45825 (local address 127.30.71.62:40725)
0810 06:39:37.113963 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.113965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.114263 (+ 298us) server_negotiation.cc:407] Connection header received
0810 06:39:37.114702 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.114704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.114716 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.114793 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:37.115855 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.116669 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.117780 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.118087 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.118165 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.120519 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:37.120521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:37.120521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:37.120529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:37.122467 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:37.122934 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:37.122935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:37.122936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:37.123033 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:37.123368 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:37.123369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:37.123370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:37.123529 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:37.123608 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.123873 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.123970 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:39:37.124162 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.113873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:40725 (local address 127.30.71.2:45825)
0810 06:39:37.114056 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.114100 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.114592 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.114854 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.114857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.115532 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:37.115734 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.115739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.116772 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.116773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.117688 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.117691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.118178 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.118179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.118283 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.118868 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:37.118876 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:37.120426 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:37.122557 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:37.122564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:37.122567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:37.122843 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:37.123115 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:37.123115 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:37.123116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:37.123268 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:37.123600 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:37.123606 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:37.123734 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.123824 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.123955 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260810 06:39:37.124728 17322 heartbeater.cc:347] Connected to a master server at 127.30.71.62:40725
I20260810 06:39:37.124837 17322 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:37.125041 17322 heartbeater.cc:511] Master 127.30.71.62:40725 requested a full tablet report, sending...
I20260810 06:39:37.125454 16975 ts_manager.cc:194] Registered new tserver with Master: ffe5c893a4884834b2c27060f05d00bb (127.30.71.2:46697)
I20260810 06:39:37.125892 16975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:45825
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:39:37.146658 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:40725
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:37.236828 17327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:37.237095 17327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:37.237141 17327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:37.238358 17327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:37.238432 17327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:37.238478 17327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:39:37.240141 17327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:40725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:37.240437 17327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:37.240695 17327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:37.246449 17335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.246541 17327 server_base.cc:1034] running on GCE node
W20260810 06:39:37.246498 17337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.246695 17333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.246930 17327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:37.247226 17327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:37.248400 17327 hybrid_clock.cc:648] HybridClock initialized: now 1786343977248384 us; error 29 us; skew 500 ppm
Aug 10 06:39:37 dist-test-slave-w206 krb5kdc[16926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:37.251238 17327 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:39:37.251727 17327 webserver.cc:460] Webserver started at http://127.30.71.3:45951/ using document root <none> and password file <none>
I20260810 06:39:37.251986 17327 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:37.252034 17327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:37.252166 17327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:37.253306 17327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "66545e38633d491b6300d35478eb8d3b"
server_key_iv: "d4e39f845a6f626ca7c636b32d147e3d"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.253656 17327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "66545e38633d491b6300d35478eb8d3b"
server_key_iv: "d4e39f845a6f626ca7c636b32d147e3d"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.255141 17327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:39:37.255987 17344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.256192 17327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:37.256325 17327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "66545e38633d491b6300d35478eb8d3b"
server_key_iv: "d4e39f845a6f626ca7c636b32d147e3d"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.256403 17327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:37.264973 17327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:37.267376 17327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:37.267542 17327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:37.267781 17327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:37.268095 17327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:37.268148 17327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.268189 17327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:37.268219 17327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.274494 17327 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:45031
I20260810 06:39:37.274564 17467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:45031 every 8 connection(s)
I20260810 06:39:37.274909 17327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
Aug 10 06:39:37 dist-test-slave-w206 krb5kdc[16926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343977, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:37.283191 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 17327
I20260810 06:39:37.283303 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:39:37.283514 31004 external_mini_cluster.cc:1428] Setting key 4c7e741249176331492af97e52c1a711
I20260810 06:39:37.285027 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.275717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:60527 (local address 127.30.71.62:40725)
0810 06:39:37.275780 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.275782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.276086 (+ 304us) server_negotiation.cc:407] Connection header received
0810 06:39:37.276491 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.276492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.276510 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.276579 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:37.277519 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.278018 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.279115 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.279328 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.279392 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.281629 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:37.281631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:37.281631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:37.281638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:37.283328 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:37.283841 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:37.283842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:37.283843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:37.283908 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:37.284380 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:37.284381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:37.284381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:37.284556 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:37.284625 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.284889 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.284962 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:39:37.285202 17470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.275692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:40725 (local address 127.30.71.3:60527)
0810 06:39:37.275875 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.275919 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.276403 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.276635 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.276638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.277225 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:37.277426 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.277430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.278103 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.278104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.278930 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.278933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.279409 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.279410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.279518 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.280007 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:37.280015 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:37.281539 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:37.283413 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:37.283421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:37.283424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:37.283757 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:37.284141 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:37.284142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:37.284143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:37.284303 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:37.284625 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:37.284631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:37.284758 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.284861 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.284992 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260810 06:39:37.285738 17468 heartbeater.cc:347] Connected to a master server at 127.30.71.62:40725
I20260810 06:39:37.285858 17468 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:37.286044 17468 heartbeater.cc:511] Master 127.30.71.62:40725 requested a full tablet report, sending...
I20260810 06:39:37.286464 16975 ts_manager.cc:194] Registered new tserver with Master: c7c6894e3a8240ba96df4ab30b4ba7d8 (127.30.71.3:45031)
I20260810 06:39:37.286819 16975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:60527
I20260810 06:39:37.295528 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:39:37.303103 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.296948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.30.71.62:40725)
0810 06:39:37.297012 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.297014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.297136 (+ 122us) server_negotiation.cc:407] Connection header received
0810 06:39:37.297176 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.297176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.297186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.297243 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:37.297772 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.298248 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.299027 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.299229 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.299283 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.300498 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:37.300500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:37.300500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:37.300506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:37.301917 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:37.302238 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:37.302238 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:37.302239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:37.302315 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:37.302533 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:37.302533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:37.302534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:37.302679 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:37.302754 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.302909 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.303038 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:39:37.304876 16975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:39:37.306454 16975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:39:37.313166 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.309094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.30.71.1:44615)
0810 06:39:37.309328 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.309335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.309434 (+ 99us) server_negotiation.cc:407] Connection header received
0810 06:39:37.309479 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.309482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.309496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.309562 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:37.309916 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.311008 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.312145 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.312715 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.312799 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.312863 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.313037 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.313078 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":107,"threads_started":1}
I20260810 06:39:37.313156 17480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.308895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44615 (local address 127.0.0.1:54654)
0810 06:39:37.309315 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.309325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.309373 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.309639 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.309640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.309642 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:37.309704 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.309706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.311142 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.311143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.312054 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.312058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.312811 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.312812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.312893 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.312906 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.313018 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.313076 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":7,"thread_start_us":70,"threads_started":1}
I20260810 06:39:37.313351 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.309249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.30.71.2:46697)
0810 06:39:37.309447 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.309451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.309462 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:39:37.309546 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.309548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.309564 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.309638 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:37.309868 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.310568 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.312078 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.312801 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.312875 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.312929 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.313199 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.313248 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
I20260810 06:39:37.313359 17479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.308904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:46697 (local address 127.0.0.1:48974)
0810 06:39:37.309380 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.309392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.309451 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.309718 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.309719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.309721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:37.309786 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.309788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.310684 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.310685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.311984 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.311987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.312967 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.312968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.313041 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.313046 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.313093 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.313138 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":104,"threads_started":1}
I20260810 06:39:37.313571 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.309171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.30.71.3:45031)
0810 06:39:37.309378 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.309383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.309393 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:39:37.309437 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.309439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.309456 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.309531 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:37.309805 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.310964 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.312326 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.313009 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.313074 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.313126 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.313402 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.313463 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
I20260810 06:39:37.313705 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.308895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:45031 (local address 127.0.0.1:52428)
0810 06:39:37.309083 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.309125 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.309196 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.309621 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.309624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.309626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:37.309704 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.309706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.311116 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.311117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.312241 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.312243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.313158 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.313159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.313214 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.313217 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.313265 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.313298 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":35,"thread_start_us":92,"threads_started":1}
I20260810 06:39:37.314159 17251 tablet_service.cc:1467] Processing CreateTablet for tablet 14277ec7fcb34ad1a01be4431c1edd34 (DEFAULT_TABLE table=test-table [id=bcdb36f84803435fb817eb8f447644a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:37.314405 17395 tablet_service.cc:1467] Processing CreateTablet for tablet 14277ec7fcb34ad1a01be4431c1edd34 (DEFAULT_TABLE table=test-table [id=bcdb36f84803435fb817eb8f447644a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:37.314441 17110 tablet_service.cc:1467] Processing CreateTablet for tablet 14277ec7fcb34ad1a01be4431c1edd34 (DEFAULT_TABLE table=test-table [id=bcdb36f84803435fb817eb8f447644a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:37.314563 17251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14277ec7fcb34ad1a01be4431c1edd34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:37.314751 17110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14277ec7fcb34ad1a01be4431c1edd34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:37.314752 17395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14277ec7fcb34ad1a01be4431c1edd34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:37.317574 17485 tablet_bootstrap.cc:492] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Bootstrap starting.
I20260810 06:39:37.318017 17488 tablet_bootstrap.cc:492] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907: Bootstrap starting.
I20260810 06:39:37.318359 17485 tablet_bootstrap.cc:654] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:37.318599 17488 tablet_bootstrap.cc:654] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:37.318923 17485 log.cc:824] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:37.319167 17487 tablet_bootstrap.cc:492] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Bootstrap starting.
I20260810 06:39:37.319170 17488 log.cc:824] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:37.319801 17485 tablet_bootstrap.cc:492] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: No bootstrap required, opened a new log
I20260810 06:39:37.319823 17487 tablet_bootstrap.cc:654] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:37.319881 17485 ts_tablet_manager.cc:1397] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:37.320407 17487 log.cc:824] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:37.321208 17487 tablet_bootstrap.cc:492] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: No bootstrap required, opened a new log
I20260810 06:39:37.321282 17487 ts_tablet_manager.cc:1397] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:37.322069 17485 raft_consensus.cc:348] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.322264 17485 raft_consensus.cc:374] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:37.322315 17485 raft_consensus.cc:729] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c6894e3a8240ba96df4ab30b4ba7d8, State: Initialized, Role: FOLLOWER
I20260810 06:39:37.322435 17485 consensus_queue.cc:260] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.322749 17485 ts_tablet_manager.cc:1428] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:37.322809 17468 heartbeater.cc:503] Master 127.30.71.62:40725 was elected leader, sending a full tablet report...
I20260810 06:39:37.323589 17487 raft_consensus.cc:348] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.323789 17487 raft_consensus.cc:374] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:37.323835 17487 raft_consensus.cc:729] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe5c893a4884834b2c27060f05d00bb, State: Initialized, Role: FOLLOWER
I20260810 06:39:37.323872 17488 tablet_bootstrap.cc:492] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907: No bootstrap required, opened a new log
I20260810 06:39:37.323967 17488 ts_tablet_manager.cc:1397] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260810 06:39:37.323961 17487 consensus_queue.cc:260] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.324275 17487 ts_tablet_manager.cc:1428] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:37.324317 17322 heartbeater.cc:503] Master 127.30.71.62:40725 was elected leader, sending a full tablet report...
I20260810 06:39:37.325842 17488 raft_consensus.cc:348] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.326009 17488 raft_consensus.cc:374] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:37.326032 17488 raft_consensus.cc:729] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7ca20bdc2d4ed2a8065e7f50deb907, State: Initialized, Role: FOLLOWER
I20260810 06:39:37.326171 17488 consensus_queue.cc:260] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.326459 17488 ts_tablet_manager.cc:1428] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:39:37.326534 17181 heartbeater.cc:503] Master 127.30.71.62:40725 was elected leader, sending a full tablet report...
I20260810 06:39:37.334260 17493 raft_consensus.cc:482] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:37.334400 17493 raft_consensus.cc:504] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.334743 17493 leader_election.cc:288] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d7ca20bdc2d4ed2a8065e7f50deb907 (127.30.71.1:44615), c7c6894e3a8240ba96df4ab30b4ba7d8 (127.30.71.3:45031)
I20260810 06:39:37.338740 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.335059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:35963 (local address 127.30.71.1:44615)
0810 06:39:37.335111 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.335113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.335124 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:39:37.335150 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.335151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.335161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.335248 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:37.335425 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.336187 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.337751 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.338332 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.338410 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.338421 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.338625 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.338665 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:39:37.338740 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.335057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:39121 (local address 127.30.71.3:45031)
0810 06:39:37.335118 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.335120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.335234 (+ 114us) server_negotiation.cc:407] Connection header received
0810 06:39:37.335304 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.335305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.335317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.335360 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:37.335517 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.336205 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.337751 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.338331 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.338410 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.338421 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.338625 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.338665 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:39:37.338763 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.334938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44615 (local address 127.30.71.2:35963)
0810 06:39:37.335002 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.335016 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.335067 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.335251 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.335252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.335254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:37.335338 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.335340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.336323 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.336324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.337650 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.337653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.338437 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.338439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.338538 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.338540 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.338610 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.338658 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":99}
I20260810 06:39:37.338775 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.334943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:45031 (local address 127.30.71.2:39121)
0810 06:39:37.335165 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.335173 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.335243 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.335381 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.335382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.335383 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:37.335446 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.335447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.336288 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.336289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.337650 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.337653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.338438 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.338439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.338532 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.338537 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.338610 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.338658 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":85,"thread_start_us":143,"threads_started":1}
I20260810 06:39:37.339188 17133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14277ec7fcb34ad1a01be4431c1edd34" candidate_uuid: "ffe5c893a4884834b2c27060f05d00bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" is_pre_election: true
I20260810 06:39:37.339237 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14277ec7fcb34ad1a01be4431c1edd34" candidate_uuid: "ffe5c893a4884834b2c27060f05d00bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" is_pre_election: true
I20260810 06:39:37.339351 17133 raft_consensus.cc:2454] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe5c893a4884834b2c27060f05d00bb in term 0.
I20260810 06:39:37.339425 17419 raft_consensus.cc:2454] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe5c893a4884834b2c27060f05d00bb in term 0.
I20260810 06:39:37.339622 17203 leader_election.cc:302] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d7ca20bdc2d4ed2a8065e7f50deb907, ffe5c893a4884834b2c27060f05d00bb; no voters:
I20260810 06:39:37.339767 17493 raft_consensus.cc:2781] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:39:37.339826 17493 raft_consensus.cc:482] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:39:37.339862 17493 raft_consensus.cc:3037] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:37.340595 17493 raft_consensus.cc:504] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.340802 17493 leader_election.cc:288] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [CANDIDATE]: Term 1 election: Requested vote from peers 9d7ca20bdc2d4ed2a8065e7f50deb907 (127.30.71.1:44615), c7c6894e3a8240ba96df4ab30b4ba7d8 (127.30.71.3:45031)
I20260810 06:39:37.340893 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14277ec7fcb34ad1a01be4431c1edd34" candidate_uuid: "ffe5c893a4884834b2c27060f05d00bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8"
I20260810 06:39:37.340982 17419 raft_consensus.cc:3037] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:37.341012 17133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14277ec7fcb34ad1a01be4431c1edd34" candidate_uuid: "ffe5c893a4884834b2c27060f05d00bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907"
I20260810 06:39:37.341084 17133 raft_consensus.cc:3037] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:37.341800 17133 raft_consensus.cc:2454] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe5c893a4884834b2c27060f05d00bb in term 1.
I20260810 06:39:37.341804 17419 raft_consensus.cc:2454] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe5c893a4884834b2c27060f05d00bb in term 1.
I20260810 06:39:37.341988 17203 leader_election.cc:302] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d7ca20bdc2d4ed2a8065e7f50deb907, ffe5c893a4884834b2c27060f05d00bb; no voters:
I20260810 06:39:37.342127 17493 raft_consensus.cc:2781] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:37.342314 17493 raft_consensus.cc:686] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 1 LEADER]: Becoming Leader. State: Replica: ffe5c893a4884834b2c27060f05d00bb, State: Running, Role: LEADER
I20260810 06:39:37.342417 17493 consensus_queue.cc:237] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } }
I20260810 06:39:37.343298 16975 catalog_manager.cc:5495] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb reported cstate change: term changed from 0 to 1, leader changed from <none> to ffe5c893a4884834b2c27060f05d00bb (127.30.71.2). New cstate: current_term: 1 leader_uuid: "ffe5c893a4884834b2c27060f05d00bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffe5c893a4884834b2c27060f05d00bb" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 46697 } health_report { overall_health: HEALTHY } } }
I20260810 06:39:37.352206 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.349598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.30.71.2:46697)
0810 06:39:37.349662 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.349664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.349683 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:39:37.349719 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.349720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.349744 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.349830 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:39:37.349971 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.350450 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.351348 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.351562 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.351645 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.351833 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:39:37.351955 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:39:37.352052 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.352071 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.352092 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:39:37.354157 17133 raft_consensus.cc:1261] T 14277ec7fcb34ad1a01be4431c1edd34 P 9d7ca20bdc2d4ed2a8065e7f50deb907 [term 1 FOLLOWER]: Refusing update from remote peer ffe5c893a4884834b2c27060f05d00bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:37.354167 17419 raft_consensus.cc:1261] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 1 FOLLOWER]: Refusing update from remote peer ffe5c893a4884834b2c27060f05d00bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:37.354516 17493 consensus_queue.cc:1035] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7ca20bdc2d4ed2a8065e7f50deb907" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 44615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:37.354682 17496 consensus_queue.cc:1035] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7c6894e3a8240ba96df4ab30b4ba7d8" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:37.357527 17501 mvcc.cc:205] Tried to move back new op lower bound from 7316864931239817216 to 7316864931195244544. Current Snapshot: MvccSnapshot[applied={T|T < 7316864931239817216}]
I20260810 06:39:37.357772 17502 mvcc.cc:205] Tried to move back new op lower bound from 7316864931239817216 to 7316864931195244544. Current Snapshot: MvccSnapshot[applied={T|T < 7316864931239817216}]
I20260810 06:39:37.360195 16975 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:37.360332 16975 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:37.361141 16975 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50f1d42c62004fab8a8d947dbfcb2f07: Sending DeleteTablet for 3 replicas of tablet 14277ec7fcb34ad1a01be4431c1edd34
I20260810 06:39:37.361510 17110 tablet_service.cc:1514] Processing DeleteTablet for tablet 14277ec7fcb34ad1a01be4431c1edd34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:37 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:54654
I20260810 06:39:37.361626 17251 tablet_service.cc:1514] Processing DeleteTablet for tablet 14277ec7fcb34ad1a01be4431c1edd34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:37 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:48974
I20260810 06:39:37.361846 17511 tablet_replica.cc:330] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: stopping tablet replica
I20260810 06:39:37.361857 17395 tablet_service.cc:1514] Processing DeleteTablet for tablet 14277ec7fcb34ad1a01be4431c1edd34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:37 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:52428
I20260810 06:39:37.362004 17511 raft_consensus.cc:2229] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:39:37.362047 17512 tablet_replica.cc:330] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: stopping tablet replica
I20260810 06:39:37.362149 17512 raft_consensus.cc:2229] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:37.362246 17511 raft_consensus.cc:2258] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:37.362250 17512 raft_consensus.cc:2258] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:37.362342 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 17044
I20260810 06:39:37.362855 17511 ts_tablet_manager.cc:1905] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:37.364987 17511 ts_tablet_manager.cc:1918] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:37.365131 17511 log.cc:1196] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/14277ec7fcb34ad1a01be4431c1edd34
I20260810 06:39:37.365698 17511 ts_tablet_manager.cc:1939] T 14277ec7fcb34ad1a01be4431c1edd34 P ffe5c893a4884834b2c27060f05d00bb: Deleting consensus metadata
I20260810 06:39:37.365921 17512 ts_tablet_manager.cc:1905] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:37.366226 16960 catalog_manager.cc:4841] TS ffe5c893a4884834b2c27060f05d00bb (127.30.71.2:46697): tablet 14277ec7fcb34ad1a01be4431c1edd34 (table test-table [id=bcdb36f84803435fb817eb8f447644a7]) successfully deleted
I20260810 06:39:37.368136 17512 ts_tablet_manager.cc:1918] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:37.368219 17512 log.cc:1196] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/14277ec7fcb34ad1a01be4431c1edd34
I20260810 06:39:37.368505 17512 ts_tablet_manager.cc:1939] T 14277ec7fcb34ad1a01be4431c1edd34 P c7c6894e3a8240ba96df4ab30b4ba7d8: Deleting consensus metadata
I20260810 06:39:37.368964 16957 catalog_manager.cc:4841] TS c7c6894e3a8240ba96df4ab30b4ba7d8 (127.30.71.3:45031): tablet 14277ec7fcb34ad1a01be4431c1edd34 (table test-table [id=bcdb36f84803435fb817eb8f447644a7]) successfully deleted
W20260810 06:39:37.371332 16958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:44615 (error 108)
I20260810 06:39:37.371907 17479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.371756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:44615 (local address 127.0.0.1:54664)
0810 06:39:37.371824 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.371865 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:39:37.372012 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 17186
W20260810 06:39:37.373953 16958 catalog_manager.cc:4584] TS 9d7ca20bdc2d4ed2a8065e7f50deb907 (127.30.71.1:44615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14277ec7fcb34ad1a01be4431c1edd34: Network error: Client connection negotiation failed: client connection to 127.30.71.1:44615: connect: Connection refused (error 111)
I20260810 06:39:37.377967 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 17327
I20260810 06:39:37.385694 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 16940
2026-08-10T06:39:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2022 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T06:39:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:37Z Disabled control of system clock
I20260810 06:39:37.416733 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:39533
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:39533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:37.505584 17518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:37.505821 17518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:37.505844 17518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:37.507144 17518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:37.507189 17518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:37.507210 17518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:37.507223 17518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:37.507236 17518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:37.508631 17518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:39533
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:39533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:37.508896 17518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:37.509233 17518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:37.514240 17524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.514364 17518 server_base.cc:1034] running on GCE node
W20260810 06:39:37.514236 17525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.514241 17527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.514765 17518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:37.515113 17518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:37.516284 17518 hybrid_clock.cc:648] HybridClock initialized: now 1786343977516282 us; error 50 us; skew 500 ppm
I20260810 06:39:37.517627 17518 webserver.cc:460] Webserver started at http://127.30.71.62:43727/ using document root <none> and password file <none>
I20260810 06:39:37.517917 17518 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:37.517992 17518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:37.518147 17518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:37.519382 17518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "06f84ad395554650a4aa37a4b4ac2856"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "d242c86c35d49f8a871bba0c85a64fcc"
server_key_iv: "a0ad9cd80c098023262eeb851693f481"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.519737 17518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "06f84ad395554650a4aa37a4b4ac2856"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "d242c86c35d49f8a871bba0c85a64fcc"
server_key_iv: "a0ad9cd80c098023262eeb851693f481"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.521109 17518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 06:39:37.521963 17533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.522140 17518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:37.522246 17518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "06f84ad395554650a4aa37a4b4ac2856"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "d242c86c35d49f8a871bba0c85a64fcc"
server_key_iv: "a0ad9cd80c098023262eeb851693f481"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.522323 17518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:37.541867 17518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:37.542254 17518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:37.542452 17518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:37.546482 17518 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:39533
I20260810 06:39:37.546581 17592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:39533 every 8 connection(s)
I20260810 06:39:37.547031 17518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:37.548112 17593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:37.550510 17593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856: Bootstrap starting.
I20260810 06:39:37.551227 17593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:37.551541 17593 log.cc:824] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:37.551914 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 17518
I20260810 06:39:37.552038 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:37.552237 31004 external_mini_cluster.cc:1428] Setting key f868e2461ffeb5a0ad319026af8c65e6
I20260810 06:39:37.552408 17593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856: No bootstrap required, opened a new log
I20260810 06:39:37.554379 17593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f84ad395554650a4aa37a4b4ac2856" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39533 } }
I20260810 06:39:37.554517 17593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:37.554540 17593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06f84ad395554650a4aa37a4b4ac2856, State: Initialized, Role: FOLLOWER
I20260810 06:39:37.554674 17593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f84ad395554650a4aa37a4b4ac2856" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39533 } }
I20260810 06:39:37.554772 17593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:37.554821 17593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:37.554872 17593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:37.555464 17593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f84ad395554650a4aa37a4b4ac2856" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39533 } }
I20260810 06:39:37.555550 17593 leader_election.cc:302] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06f84ad395554650a4aa37a4b4ac2856; no voters:
I20260810 06:39:37.555754 17593 leader_election.cc:288] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:37.555786 17599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:37.555896 17599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [term 1 LEADER]: Becoming Leader. State: Replica: 06f84ad395554650a4aa37a4b4ac2856, State: Running, Role: LEADER
I20260810 06:39:37.556030 17593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:37.556034 17599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f84ad395554650a4aa37a4b4ac2856" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39533 } }
I20260810 06:39:37.556492 17599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06f84ad395554650a4aa37a4b4ac2856. Latest consensus state: current_term: 1 leader_uuid: "06f84ad395554650a4aa37a4b4ac2856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f84ad395554650a4aa37a4b4ac2856" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39533 } } }
I20260810 06:39:37.556502 17600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06f84ad395554650a4aa37a4b4ac2856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06f84ad395554650a4aa37a4b4ac2856" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39533 } } }
I20260810 06:39:37.556690 17600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:37.556901 17599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:37.557291 17610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:37.557842 17610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:37.559394 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.552794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.30.71.62:39533)
0810 06:39:37.553032 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.553041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.553075 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:39:37.553718 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.553725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.553785 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.553939 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:37.554148 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.555920 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.557060 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.557692 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.557839 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.558605 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:37.558609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:37.558609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:37.558817 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:37.558858 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:37.558873 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:37.558968 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.559090 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.559175 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260810 06:39:37.559669 17610 catalog_manager.cc:1347] Generated new cluster ID: e0bf102c979c4f6abd4b5de97a352581
I20260810 06:39:37.559791 17610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:37.567399 17610 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:37.568135 17610 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:37.579615 17610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856: Generated new TSK 0
I20260810 06:39:37.579849 17610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:37.590525 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39533
--builtin_ntp_servers=127.30.71.20:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:37.676236 17621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:37.676491 17621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:37.676529 17621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:37.677788 17621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:37.677875 17621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:37.677922 17621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:39:37.679420 17621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:37.679693 17621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:37.680043 17621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:37.686378 17630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.686389 17628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.686555 17621 server_base.cc:1034] running on GCE node
W20260810 06:39:37.686381 17627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.686839 17621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:37.687137 17621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:37.688328 17621 hybrid_clock.cc:648] HybridClock initialized: now 1786343977688307 us; error 33 us; skew 500 ppm
I20260810 06:39:37.689824 17621 webserver.cc:460] Webserver started at http://127.30.71.1:35909/ using document root <none> and password file <none>
I20260810 06:39:37.690104 17621 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:37.690163 17621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:37.690305 17621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:37.691754 17621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "b99c04db02b14343a75b7637f3ec663d"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "93aec104d1b1e71fe90190d7e51ddad6"
server_key_iv: "bb7ded090e2a42e21580d92693987ab3"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.692158 17621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "b99c04db02b14343a75b7637f3ec663d"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "93aec104d1b1e71fe90190d7e51ddad6"
server_key_iv: "bb7ded090e2a42e21580d92693987ab3"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.693581 17621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:39:37.694396 17637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.694607 17621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:37.694722 17621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "b99c04db02b14343a75b7637f3ec663d"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "93aec104d1b1e71fe90190d7e51ddad6"
server_key_iv: "bb7ded090e2a42e21580d92693987ab3"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.694797 17621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:37.711127 17621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:37.711491 17621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:37.711647 17621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:37.711891 17621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:37.712237 17621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:37.712302 17621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.712368 17621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:37.712399 17621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.718129 17621 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38295
I20260810 06:39:37.718240 17763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38295 every 8 connection(s)
I20260810 06:39:37.718597 17621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:39:37.723893 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.719433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:36399 (local address 127.30.71.62:39533)
0810 06:39:37.719501 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.719502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.719724 (+ 222us) server_negotiation.cc:407] Connection header received
0810 06:39:37.720211 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.720212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.720227 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.720291 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:37.720928 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.721421 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.722422 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.722624 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.722693 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.723650 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:37.723651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:37.723651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:37.723656 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:37.723671 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:37.723675 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:37.723722 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.723809 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.723833 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:39:37.724141 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.719408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39533 (local address 127.30.71.1:36399)
0810 06:39:37.719596 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.719638 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.720127 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.720342 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.720346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.720354 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:37.720852 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.720856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.721507 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.721507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.722306 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.722308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.722738 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.722739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.722870 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.723507 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:37.723508 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:37.723538 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:37.723561 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:37.723733 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:37.723738 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:37.723741 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.723796 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.723879 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260810 06:39:37.724807 17764 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39533
I20260810 06:39:37.725023 17764 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:37.725237 17764 heartbeater.cc:511] Master 127.30.71.62:39533 requested a full tablet report, sending...
I20260810 06:39:37.725862 17550 ts_manager.cc:194] Registered new tserver with Master: b99c04db02b14343a75b7637f3ec663d (127.30.71.1:38295)
I20260810 06:39:37.726244 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 17621
I20260810 06:39:37.726364 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:39:37.726578 31004 external_mini_cluster.cc:1428] Setting key b984eb2efb9bcd35c32bbafdcf37f0fc
I20260810 06:39:37.726676 17550 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.1:36399
I20260810 06:39:37.727561 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39533
--builtin_ntp_servers=127.30.71.20:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:37.811687 17767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:37.811908 17767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:37.811930 17767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:37.813123 17767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:37.813169 17767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:37.813194 17767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:39:37.814479 17767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:37.814675 17767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:37.814929 17767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:37.820165 17773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.820218 17767 server_base.cc:1034] running on GCE node
W20260810 06:39:37.820173 17774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.820354 17776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.820595 17767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:37.820813 17767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:37.821954 17767 hybrid_clock.cc:648] HybridClock initialized: now 1786343977821933 us; error 36 us; skew 500 ppm
I20260810 06:39:37.823239 17767 webserver.cc:460] Webserver started at http://127.30.71.2:43363/ using document root <none> and password file <none>
I20260810 06:39:37.823514 17767 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:37.823614 17767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:37.823770 17767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:37.824859 17767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "d9a1343dc6a84c01834fa59399b44167"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "6fa686d9156c4a34acb5cb24a4118d89"
server_key_iv: "f718ed31f5ee8c13d8ca2b6bd199269a"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.825238 17767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "d9a1343dc6a84c01834fa59399b44167"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "6fa686d9156c4a34acb5cb24a4118d89"
server_key_iv: "f718ed31f5ee8c13d8ca2b6bd199269a"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.826711 17767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:37.827566 17783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.827788 17767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.827891 17767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "d9a1343dc6a84c01834fa59399b44167"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "6fa686d9156c4a34acb5cb24a4118d89"
server_key_iv: "f718ed31f5ee8c13d8ca2b6bd199269a"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.827981 17767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:37.840368 17767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:37.840718 17767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:37.840875 17767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:37.841110 17767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:37.841456 17767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:37.841512 17767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.841554 17767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:37.841583 17767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.847905 17767 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:39251
I20260810 06:39:37.847965 17905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:39251 every 8 connection(s)
I20260810 06:39:37.848332 17767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:39:37.853101 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 17767
I20260810 06:39:37.853197 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:39:37.853426 31004 external_mini_cluster.cc:1428] Setting key 458cacf33f46601e869fe10e8e3ba7a3
I20260810 06:39:37.853978 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.849119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:43943 (local address 127.30.71.62:39533)
0810 06:39:37.849188 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:37.849191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:37.849468 (+ 277us) server_negotiation.cc:407] Connection header received
0810 06:39:37.850010 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:37.850010 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:37.850023 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:37.850077 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:37.850787 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.851348 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.852340 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.852537 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.852615 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.853616 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:37.853618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:37.853619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:37.853625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:37.853646 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:37.853651 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:37.853724 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:37.853866 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 06:39:37.853895 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:39:37.854130 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:37.849087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39533 (local address 127.30.71.2:43943)
0810 06:39:37.849328 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:37.849364 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:37.849896 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:37.850163 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:37.850166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:37.850175 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:37.850680 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.850684 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.851458 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.851459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.852230 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:37.852232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:37.852623 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:37.852625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:37.852761 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:37.853439 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:37.853440 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:37.853481 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:37.853511 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:37.853751 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:37.853759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:37.853762 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:37.853838 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 06:39:37.853932 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260810 06:39:37.854609 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39533
--builtin_ntp_servers=127.30.71.20:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:39:37.854658 17906 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39533
I20260810 06:39:37.854751 17906 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:37.854940 17906 heartbeater.cc:511] Master 127.30.71.62:39533 requested a full tablet report, sending...
I20260810 06:39:37.855427 17550 ts_manager.cc:194] Registered new tserver with Master: d9a1343dc6a84c01834fa59399b44167 (127.30.71.2:39251)
I20260810 06:39:37.855852 17550 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.2:43943
W20260810 06:39:37.937511 17909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:37.937757 17909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:37.937822 17909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:37.938990 17909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:37.939059 17909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:37.939101 17909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:39:37.940472 17909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:37.940748 17909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:37.941063 17909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:37.946276 17918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.946329 17915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:37.946353 17916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:37.946660 17909 server_base.cc:1034] running on GCE node
I20260810 06:39:37.946833 17909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:37.947124 17909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:37.948266 17909 hybrid_clock.cc:648] HybridClock initialized: now 1786343977948244 us; error 36 us; skew 500 ppm
I20260810 06:39:37.949522 17909 webserver.cc:460] Webserver started at http://127.30.71.3:38469/ using document root <none> and password file <none>
I20260810 06:39:37.949813 17909 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:37.949864 17909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:37.950054 17909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:37.951262 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "443e353d274847d69b19ef4bde65c976"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "bc8f07593d412c30e6cf5bd680f78509"
server_key_iv: "80cdee6137de84869cc06e8da9a05579"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.951607 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "443e353d274847d69b19ef4bde65c976"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "bc8f07593d412c30e6cf5bd680f78509"
server_key_iv: "80cdee6137de84869cc06e8da9a05579"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.952921 17909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:37.953684 17926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:37.953881 17909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:39:37.953996 17909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "443e353d274847d69b19ef4bde65c976"
format_stamp: "Formatted at 2026-08-10 06:39:37 on dist-test-slave-w206"
server_key: "bc8f07593d412c30e6cf5bd680f78509"
server_key_iv: "80cdee6137de84869cc06e8da9a05579"
server_key_version: "encryptionkey@0"
I20260810 06:39:37.954074 17909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:38.000911 17909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:38.001279 17909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:38.001456 17909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:38.001735 17909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:38.002120 17909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:38.002177 17909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:38.002213 17909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:38.002230 17909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:38.008586 17909 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:39875
I20260810 06:39:38.008668 18047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:39875 every 8 connection(s)
I20260810 06:39:38.009027 17909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:39:38.011189 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 17909
I20260810 06:39:38.011310 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:39:38.011525 31004 external_mini_cluster.cc:1428] Setting key 96a52d73176b061acce571fcaaddaf23
I20260810 06:39:38.014623 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.009829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:36947 (local address 127.30.71.62:39533)
0810 06:39:38.009902 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.009904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.010180 (+ 276us) server_negotiation.cc:407] Connection header received
0810 06:39:38.010726 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.010727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.010739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.010802 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:38.011609 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.012119 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.013160 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.013376 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.013459 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.014283 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:38.014284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:38.014285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:38.014290 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:38.014307 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:38.014311 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:38.014395 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.014515 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.014543 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:39:38.014882 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.009852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39533 (local address 127.30.71.3:36947)
0810 06:39:38.010053 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:38.010089 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:38.010613 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:38.010890 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:38.010893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:38.010901 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:38.011515 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.011521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.012231 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.012232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.013065 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.013067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.013467 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.013469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.013607 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.014141 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:38.014141 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:38.014170 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:38.014193 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:38.014404 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:38.014410 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:38.014414 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:39:38.014511 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:39:38.014619 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260810 06:39:38.015584 18048 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39533
I20260810 06:39:38.015674 18048 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:38.015901 18048 heartbeater.cc:511] Master 127.30.71.62:39533 requested a full tablet report, sending...
I20260810 06:39:38.016323 17550 ts_manager.cc:194] Registered new tserver with Master: 443e353d274847d69b19ef4bde65c976 (127.30.71.3:39875)
I20260810 06:39:38.016709 17550 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.3:36947
I20260810 06:39:38.023468 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:39:38.027647 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.024930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.30.71.62:39533)
0810 06:39:38.025014 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.025016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.025108 (+ 92us) server_negotiation.cc:407] Connection header received
0810 06:39:38.025147 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.025148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.025156 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.025212 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:38.025377 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.025840 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.026573 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.026752 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.026813 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.027461 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:38.027462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:38.027462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:38.027466 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:38.027480 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:38.027483 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:38.027526 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.027574 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.027592 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:39:38.029315 17550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:39:38.030867 17550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:39:38.037066 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.033192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:39251 (local address 127.0.0.1:52244)
0810 06:39:38.033583 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:38.033594 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:38.033642 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:38.033969 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:38.033970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:38.033972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:38.034016 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.034017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.034847 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.034848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.036043 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.036046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.036843 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.036844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.036902 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.036914 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:39:38.036967 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 06:39:38.037007 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":38,"thread_start_us":72,"threads_started":1}
I20260810 06:39:38.037113 18062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.033403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52244 (local address 127.30.71.2:39251)
0810 06:39:38.033571 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.033575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.033765 (+ 190us) server_negotiation.cc:407] Connection header received
0810 06:39:38.033875 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.033878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.033893 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.033960 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:38.034069 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.034753 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.036130 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.036761 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.036838 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.036931 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.036992 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.037027 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260810 06:39:38.037613 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.032986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39875 (local address 127.0.0.1:33726)
0810 06:39:38.033290 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:38.033334 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:38.033413 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:38.033774 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:38.033775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:38.033777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:38.033823 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.033824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.035168 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.035169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.036385 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.036388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.037371 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.037372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.037420 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.037424 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:39:38.037497 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:39:38.037542 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":77,"thread_start_us":74,"threads_started":1}
I20260810 06:39:38.037999 17833 tablet_service.cc:1467] Processing CreateTablet for tablet cdead201954d45fbb4c70d22b92f49aa (DEFAULT_TABLE table=test-table [id=a2376a388e4f4313bcbf02e24c9565c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:38.038407 17833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdead201954d45fbb4c70d22b92f49aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:38.038753 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.032988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38295 (local address 127.0.0.1:53704)
0810 06:39:38.033305 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:38.033341 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:38.033414 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:38.033672 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:38.033675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:38.033677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:38.033746 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.033748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.035168 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.035169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.036385 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.036388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.038574 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.038575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.038623 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.038626 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:38.038672 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:39:38.038703 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":38,"thread_start_us":58,"threads_started":1}
I20260810 06:39:38.038846 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.033083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.30.71.1:38295)
0810 06:39:38.033344 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.033348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.033420 (+ 72us) server_negotiation.cc:407] Connection header received
0810 06:39:38.033541 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.033544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.033563 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.033649 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:38.033847 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.035070 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.037721 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.038453 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.038530 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.038616 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.038687 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.038725 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":106,"threads_started":1}
I20260810 06:39:38.039573 17689 tablet_service.cc:1467] Processing CreateTablet for tablet cdead201954d45fbb4c70d22b92f49aa (DEFAULT_TABLE table=test-table [id=a2376a388e4f4313bcbf02e24c9565c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:38.039963 17689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdead201954d45fbb4c70d22b92f49aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:38.042035 18063 tablet_bootstrap.cc:492] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Bootstrap starting.
I20260810 06:39:38.042174 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.033076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.30.71.3:39875)
0810 06:39:38.033254 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.033260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.033396 (+ 136us) server_negotiation.cc:407] Connection header received
0810 06:39:38.033503 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.033505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.033522 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.033717 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:38.033903 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.035069 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.036481 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.037187 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.037257 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.037340 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.042026 (+ 4686us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.042081 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260810 06:39:38.042877 18063 tablet_bootstrap.cc:654] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:38.042922 17976 tablet_service.cc:1467] Processing CreateTablet for tablet cdead201954d45fbb4c70d22b92f49aa (DEFAULT_TABLE table=test-table [id=a2376a388e4f4313bcbf02e24c9565c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:38.043279 18063 log.cc:824] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:38.043313 17976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdead201954d45fbb4c70d22b92f49aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:38.044018 18063 tablet_bootstrap.cc:492] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: No bootstrap required, opened a new log
I20260810 06:39:38.044075 18063 ts_tablet_manager.cc:1397] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:39:38.045974 18066 tablet_bootstrap.cc:492] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Bootstrap starting.
I20260810 06:39:38.046057 18064 tablet_bootstrap.cc:492] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: Bootstrap starting.
I20260810 06:39:38.046139 18063 raft_consensus.cc:348] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.046324 18063 raft_consensus.cc:374] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:38.046398 18063 raft_consensus.cc:729] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a1343dc6a84c01834fa59399b44167, State: Initialized, Role: FOLLOWER
I20260810 06:39:38.046541 18063 consensus_queue.cc:260] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.046646 18066 tablet_bootstrap.cc:654] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:38.046732 18064 tablet_bootstrap.cc:654] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:38.046859 18063 ts_tablet_manager.cc:1428] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:38.047000 18066 log.cc:824] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:38.047039 17906 heartbeater.cc:503] Master 127.30.71.62:39533 was elected leader, sending a full tablet report...
I20260810 06:39:38.047093 18064 log.cc:824] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:38.047693 18066 tablet_bootstrap.cc:492] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: No bootstrap required, opened a new log
I20260810 06:39:38.047776 18066 ts_tablet_manager.cc:1397] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:39:38.049764 18066 raft_consensus.cc:348] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.049916 18066 raft_consensus.cc:374] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:38.049961 18066 raft_consensus.cc:729] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443e353d274847d69b19ef4bde65c976, State: Initialized, Role: FOLLOWER
I20260810 06:39:38.050081 18066 consensus_queue.cc:260] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.050354 18066 ts_tablet_manager.cc:1428] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:38.050410 18048 heartbeater.cc:503] Master 127.30.71.62:39533 was elected leader, sending a full tablet report...
I20260810 06:39:38.050927 18064 tablet_bootstrap.cc:492] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: No bootstrap required, opened a new log
I20260810 06:39:38.051028 18064 ts_tablet_manager.cc:1397] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:39:38.052954 18064 raft_consensus.cc:348] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.053078 18064 raft_consensus.cc:374] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:38.053098 18064 raft_consensus.cc:729] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b99c04db02b14343a75b7637f3ec663d, State: Initialized, Role: FOLLOWER
I20260810 06:39:38.053201 18064 consensus_queue.cc:260] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.053421 18064 ts_tablet_manager.cc:1428] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:38.053488 17764 heartbeater.cc:503] Master 127.30.71.62:39533 was elected leader, sending a full tablet report...
W20260810 06:39:38.098896 17907 tablet.cc:2369] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:39:38.219327 17765 tablet.cc:2369] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:38.249384 18067 raft_consensus.cc:482] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:38.249516 18067 raft_consensus.cc:504] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.249927 18067 leader_election.cc:288] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 443e353d274847d69b19ef4bde65c976 (127.30.71.3:39875), b99c04db02b14343a75b7637f3ec663d (127.30.71.1:38295)
I20260810 06:39:38.253214 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.250171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38295 (local address 127.30.71.2:52459)
0810 06:39:38.250284 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:38.250299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:38.250352 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:38.250534 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:38.250535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:38.250537 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:38.250616 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.250618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.251422 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.251423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.252296 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.252299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.252924 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.252925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.253000 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.253004 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:39:38.253075 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:39:38.253121 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 06:39:38.253243 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.250258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:52459 (local address 127.30.71.1:38295)
0810 06:39:38.250326 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.250328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.250342 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:39:38.250433 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.250434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.250443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.250508 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:38.250723 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.251341 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.252393 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.252830 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.252885 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.252894 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.253104 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.253142 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:39:38.253578 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.250258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:35955 (local address 127.30.71.3:39875)
0810 06:39:38.250360 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.250362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.250676 (+ 314us) server_negotiation.cc:407] Connection header received
0810 06:39:38.250738 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.250739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.250754 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.250812 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:38.251015 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.251607 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.252622 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.253111 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.253176 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.253186 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.253423 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.253502 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:39:38.253690 17713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdead201954d45fbb4c70d22b92f49aa" candidate_uuid: "d9a1343dc6a84c01834fa59399b44167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b99c04db02b14343a75b7637f3ec663d" is_pre_election: true
I20260810 06:39:38.253824 18072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.250181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:39875 (local address 127.30.71.2:35955)
0810 06:39:38.250601 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:38.250611 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:38.250656 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:38.250829 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:38.250829 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:38.250831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:38.250899 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.250900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.251693 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.251694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.252520 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:38.252523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.253211 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.253212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:38.253300 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.253305 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:38.253684 (+ 379us) client_negotiation.cc:240] Negotiation successful
0810 06:39:38.253734 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":248,"thread_start_us":117,"threads_started":1}
I20260810 06:39:38.253899 17713 raft_consensus.cc:2454] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a1343dc6a84c01834fa59399b44167 in term 0.
I20260810 06:39:38.254181 17785 leader_election.cc:302] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b99c04db02b14343a75b7637f3ec663d, d9a1343dc6a84c01834fa59399b44167; no voters:
I20260810 06:39:38.254211 17997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdead201954d45fbb4c70d22b92f49aa" candidate_uuid: "d9a1343dc6a84c01834fa59399b44167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e353d274847d69b19ef4bde65c976" is_pre_election: true
I20260810 06:39:38.254307 18067 raft_consensus.cc:2781] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:39:38.254372 18067 raft_consensus.cc:482] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:39:38.254390 17997 raft_consensus.cc:2454] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a1343dc6a84c01834fa59399b44167 in term 0.
I20260810 06:39:38.254403 18067 raft_consensus.cc:3037] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:38.255280 18067 raft_consensus.cc:504] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.255445 18067 leader_election.cc:288] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [CANDIDATE]: Term 1 election: Requested vote from peers 443e353d274847d69b19ef4bde65c976 (127.30.71.3:39875), b99c04db02b14343a75b7637f3ec663d (127.30.71.1:38295)
I20260810 06:39:38.255581 17997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdead201954d45fbb4c70d22b92f49aa" candidate_uuid: "d9a1343dc6a84c01834fa59399b44167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e353d274847d69b19ef4bde65c976"
I20260810 06:39:38.255652 17713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdead201954d45fbb4c70d22b92f49aa" candidate_uuid: "d9a1343dc6a84c01834fa59399b44167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b99c04db02b14343a75b7637f3ec663d"
I20260810 06:39:38.255667 17997 raft_consensus.cc:3037] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:38.255752 17713 raft_consensus.cc:3037] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:38.256691 17997 raft_consensus.cc:2454] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a1343dc6a84c01834fa59399b44167 in term 1.
I20260810 06:39:38.256691 17713 raft_consensus.cc:2454] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a1343dc6a84c01834fa59399b44167 in term 1.
I20260810 06:39:38.256919 17786 leader_election.cc:302] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 443e353d274847d69b19ef4bde65c976, d9a1343dc6a84c01834fa59399b44167; no voters:
I20260810 06:39:38.257023 18067 raft_consensus.cc:2781] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:38.257179 18067 raft_consensus.cc:686] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 1 LEADER]: Becoming Leader. State: Replica: d9a1343dc6a84c01834fa59399b44167, State: Running, Role: LEADER
I20260810 06:39:38.257283 18067 consensus_queue.cc:237] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } }
I20260810 06:39:38.258173 17550 catalog_manager.cc:5495] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9a1343dc6a84c01834fa59399b44167 (127.30.71.2). New cstate: current_term: 1 leader_uuid: "d9a1343dc6a84c01834fa59399b44167" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9a1343dc6a84c01834fa59399b44167" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39251 } health_report { overall_health: HEALTHY } } }
W20260810 06:39:38.259594 18049 tablet.cc:2369] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:38.263288 18062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.261094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.30.71.2:39251)
0810 06:39:38.261132 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:38.261133 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:38.261142 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:39:38.261211 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:38.261212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:38.261228 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:38.261291 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:39:38.261420 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.261872 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.262589 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:38.262762 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:38.262831 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:38.262937 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:39:38.263097 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:39:38.263161 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:38.263194 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 06:39:38.263213 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 06:39:38.265178 17713 raft_consensus.cc:1261] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 1 FOLLOWER]: Refusing update from remote peer d9a1343dc6a84c01834fa59399b44167: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:38.265177 17997 raft_consensus.cc:1261] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 1 FOLLOWER]: Refusing update from remote peer d9a1343dc6a84c01834fa59399b44167: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:38.265553 18067 consensus_queue.cc:1035] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b99c04db02b14343a75b7637f3ec663d" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:38.265722 18073 consensus_queue.cc:1035] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443e353d274847d69b19ef4bde65c976" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 39875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:38.271082 17550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:38.271188 17550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:38.272058 17550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 06f84ad395554650a4aa37a4b4ac2856: Sending DeleteTablet for 3 replicas of tablet cdead201954d45fbb4c70d22b92f49aa
I20260810 06:39:38.272433 17976 tablet_service.cc:1514] Processing DeleteTablet for tablet cdead201954d45fbb4c70d22b92f49aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:38 UTC) from {username='slave'} at 127.0.0.1:33726
I20260810 06:39:38.272461 17833 tablet_service.cc:1514] Processing DeleteTablet for tablet cdead201954d45fbb4c70d22b92f49aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:38 UTC) from {username='slave'} at 127.0.0.1:52244
I20260810 06:39:38.272503 17689 tablet_service.cc:1514] Processing DeleteTablet for tablet cdead201954d45fbb4c70d22b92f49aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:38 UTC) from {username='slave'} at 127.0.0.1:53704
I20260810 06:39:38.272783 18091 tablet_replica.cc:330] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d: stopping tablet replica
I20260810 06:39:38.272840 18090 tablet_replica.cc:330] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: stopping tablet replica
I20260810 06:39:38.272953 18090 raft_consensus.cc:2229] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:39:38.273032 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 17621
I20260810 06:39:38.273082 18090 raft_consensus.cc:2258] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:38.273200 18091 raft_consensus.cc:2229] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:38.273393 18091 raft_consensus.cc:2258] T cdead201954d45fbb4c70d22b92f49aa P b99c04db02b14343a75b7637f3ec663d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:38.273546 18090 ts_tablet_manager.cc:1905] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:38.273684 18089 tablet_replica.cc:330] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: stopping tablet replica
I20260810 06:39:38.273823 18089 raft_consensus.cc:2229] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:38.273975 18089 raft_consensus.cc:2258] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:38.274668 18089 ts_tablet_manager.cc:1905] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:38.276047 18090 ts_tablet_manager.cc:1918] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:38.276156 18090 log.cc:1196] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/cdead201954d45fbb4c70d22b92f49aa
I20260810 06:39:38.276508 18090 ts_tablet_manager.cc:1939] T cdead201954d45fbb4c70d22b92f49aa P d9a1343dc6a84c01834fa59399b44167: Deleting consensus metadata
I20260810 06:39:38.276680 18089 ts_tablet_manager.cc:1918] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:38.276813 18089 log.cc:1196] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/cdead201954d45fbb4c70d22b92f49aa
I20260810 06:39:38.277069 17536 catalog_manager.cc:4841] TS d9a1343dc6a84c01834fa59399b44167 (127.30.71.2:39251): tablet cdead201954d45fbb4c70d22b92f49aa (table test-table [id=a2376a388e4f4313bcbf02e24c9565c6]) successfully deleted
I20260810 06:39:38.277134 18089 ts_tablet_manager.cc:1939] T cdead201954d45fbb4c70d22b92f49aa P 443e353d274847d69b19ef4bde65c976: Deleting consensus metadata
I20260810 06:39:38.277701 17536 catalog_manager.cc:4841] TS 443e353d274847d69b19ef4bde65c976 (127.30.71.3:39875): tablet cdead201954d45fbb4c70d22b92f49aa (table test-table [id=a2376a388e4f4313bcbf02e24c9565c6]) successfully deleted
W20260810 06:39:38.285064 17535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:38295 (error 108)
I20260810 06:39:38.285281 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 17767
I20260810 06:39:38.287751 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:38.287593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38295 (local address 127.0.0.1:53716)
0810 06:39:38.287651 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:38.287701 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:39:38.288053 17535 catalog_manager.cc:4584] TS b99c04db02b14343a75b7637f3ec663d (127.30.71.1:38295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cdead201954d45fbb4c70d22b92f49aa: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38295: connect: Connection refused (error 111)
I20260810 06:39:38.292058 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 17909
I20260810 06:39:38.298197 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 17518
2026-08-10T06:39:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (911 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:38 dist-test-slave-w206 krb5kdc[18095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:38 dist-test-slave-w206 krb5kdc[18095](info): set up 2 sockets
Aug 10 06:39:38 dist-test-slave-w206 krb5kdc[18095](info): commencing operation
krb5kdc: starting...
W20260810 06:39:39.377665 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:39 dist-test-slave-w206 krb5kdc[18095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:39:39.506618 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37099
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:37099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:39.591871 18108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:39.592140 18108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:39.592183 18108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:39.593426 18108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:39.593500 18108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:39.593528 18108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:39.593561 18108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:39.593587 18108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:39.595216 18108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:37099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:37099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:39.595506 18108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:39.595745 18108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:39.601013 18114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:39.601022 18113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:39.601022 18116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:39.601082 18108 server_base.cc:1034] running on GCE node
I20260810 06:39:39.601511 18108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:39.601766 18108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:39.602944 18108 hybrid_clock.cc:648] HybridClock initialized: now 1786343979602929 us; error 26 us; skew 500 ppm
Aug 10 06:39:39 dist-test-slave-w206 krb5kdc[18095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:39.605587 18108 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:39:39.606060 18108 webserver.cc:460] Webserver started at http://127.30.71.62:45473/ using document root <none> and password file <none>
I20260810 06:39:39.606313 18108 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:39.606384 18108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:39.606539 18108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:39.607715 18108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "190931ede35842a6a9efd387caa02354"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "e8bec50be754663759ff5075b62bb5e4"
server_key_iv: "e2f3d3b99de1372b0d47457e313d328f"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.608098 18108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "190931ede35842a6a9efd387caa02354"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "e8bec50be754663759ff5075b62bb5e4"
server_key_iv: "e2f3d3b99de1372b0d47457e313d328f"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.609524 18108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:39:39.610272 18123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:39.610529 18108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:39.610671 18108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "190931ede35842a6a9efd387caa02354"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "e8bec50be754663759ff5075b62bb5e4"
server_key_iv: "e2f3d3b99de1372b0d47457e313d328f"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.610759 18108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:39.632136 18108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:39.634568 18108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:39.634766 18108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:39.638805 18108 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:37099
I20260810 06:39:39.638880 18179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:37099 every 8 connection(s)
I20260810 06:39:39.639321 18108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:39.640302 18180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:39.642777 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 18108
I20260810 06:39:39.642922 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:39.643025 18180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354: Bootstrap starting.
I20260810 06:39:39.643190 31004 external_mini_cluster.cc:1428] Setting key c294ef21cd7e4c1d73d57a5f9c019fce
I20260810 06:39:39.643793 18180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:39.644127 18180 log.cc:824] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:39.644912 18180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354: No bootstrap required, opened a new log
I20260810 06:39:39.646934 18180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190931ede35842a6a9efd387caa02354" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37099 } }
I20260810 06:39:39.647164 18180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:39.647215 18180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190931ede35842a6a9efd387caa02354, State: Initialized, Role: FOLLOWER
I20260810 06:39:39.647389 18180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190931ede35842a6a9efd387caa02354" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37099 } }
I20260810 06:39:39.647490 18180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:39.647534 18180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:39.647583 18180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:39.648178 18180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190931ede35842a6a9efd387caa02354" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37099 } }
I20260810 06:39:39.648301 18180 leader_election.cc:302] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 190931ede35842a6a9efd387caa02354; no voters:
I20260810 06:39:39.648486 18180 leader_election.cc:288] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:39.648547 18185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:39.648720 18185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [term 1 LEADER]: Becoming Leader. State: Replica: 190931ede35842a6a9efd387caa02354, State: Running, Role: LEADER
I20260810 06:39:39.648762 18180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:39.648854 18185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190931ede35842a6a9efd387caa02354" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37099 } }
I20260810 06:39:39.649242 18185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 190931ede35842a6a9efd387caa02354. Latest consensus state: current_term: 1 leader_uuid: "190931ede35842a6a9efd387caa02354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190931ede35842a6a9efd387caa02354" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37099 } } }
I20260810 06:39:39.649256 18186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "190931ede35842a6a9efd387caa02354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190931ede35842a6a9efd387caa02354" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 37099 } } }
I20260810 06:39:39.649339 18185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:39.649354 18186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:39.650367 18196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 06:39:39 dist-test-slave-w206 krb5kdc[18095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:39.650933 18196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:39.652908 18196 catalog_manager.cc:1347] Generated new cluster ID: 3eb2767799144c3c84004dc51f0c4697
I20260810 06:39:39.652961 18196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:39.655552 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:39.643817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.30.71.62:37099)
0810 06:39:39.644016 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:39.644034 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:39.644060 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:39:39.644667 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:39.644675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:39.644724 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:39.644903 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:39.645568 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:39.646494 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:39.647442 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:39.647982 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:39.648106 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:39.651292 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:39.651296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:39.651297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:39.651316 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:39.653758 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:39.654232 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:39.654235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:39.654236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:39.654341 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:39.654581 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:39.654582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:39.654582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:39.654876 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:39.655055 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:39.655210 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:39:39.655348 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260810 06:39:39.657794 18196 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:39.658279 18196 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:39.664577 18196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354: Generated new TSK 0
I20260810 06:39:39.664789 18196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:39:39.697660 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37099
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:39.782820 18209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:39.783067 18209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:39.783097 18209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:39.784336 18209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:39.784384 18209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:39.784412 18209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:39:39.785923 18209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:39.786178 18209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:39.786449 18209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:39.791775 18217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:39.791782 18215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:39.791808 18214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:39.791929 18209 server_base.cc:1034] running on GCE node
I20260810 06:39:39.792346 18209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:39.792608 18209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:39.793783 18209 hybrid_clock.cc:648] HybridClock initialized: now 1786343979793764 us; error 39 us; skew 500 ppm
Aug 10 06:39:39 dist-test-slave-w206 krb5kdc[18095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:39.796586 18209 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:39:39.797101 18209 webserver.cc:460] Webserver started at http://127.30.71.1:42643/ using document root <none> and password file <none>
I20260810 06:39:39.797353 18209 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:39.797423 18209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:39.797575 18209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:39.798682 18209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "0a9e280b6c8b49e2a1104133a765d1df"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "eb5c492c62a451a6ec3323fd7449d991"
server_key_iv: "ccc3d11cc5ab0c06059229fb75649183"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.799144 18209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "0a9e280b6c8b49e2a1104133a765d1df"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "eb5c492c62a451a6ec3323fd7449d991"
server_key_iv: "ccc3d11cc5ab0c06059229fb75649183"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.800704 18209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:39.801720 18225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:39.801936 18209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:39.802035 18209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "0a9e280b6c8b49e2a1104133a765d1df"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "eb5c492c62a451a6ec3323fd7449d991"
server_key_iv: "ccc3d11cc5ab0c06059229fb75649183"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.802129 18209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:39.815100 18209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:39.817206 18209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:39.817399 18209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:39.817654 18209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:39.817981 18209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:39.818035 18209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:39.818081 18209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:39.818107 18209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:39.824101 18209 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:45977
I20260810 06:39:39.824184 18345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:45977 every 8 connection(s)
I20260810 06:39:39.824540 18209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
Aug 10 06:39:39 dist-test-slave-w206 krb5kdc[18095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:39.834029 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 18209
I20260810 06:39:39.834169 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:39:39.834375 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:39.825351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:58221 (local address 127.30.71.62:37099)
0810 06:39:39.825447 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:39.825450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:39.825624 (+ 174us) server_negotiation.cc:407] Connection header received
0810 06:39:39.826638 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:39.826640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:39.826652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:39.826702 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:39.827389 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:39.827906 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:39.828958 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:39.829198 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:39.829287 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:39.831580 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:39.831582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:39.831582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:39.831590 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:39.832917 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:39.833350 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:39.833351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:39.833352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:39.833420 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:39.833676 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:39.833677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:39.833677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:39.833802 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:39.833878 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:39.834176 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 06:39:39.834278 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:39:39.834393 31004 external_mini_cluster.cc:1428] Setting key c1766306488e7b8cc61909d75e63f3bb
I20260810 06:39:39.834609 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:39.825302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37099 (local address 127.30.71.1:58221)
0810 06:39:39.825499 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:39.825536 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:39.826519 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:39.826799 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:39.826803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:39.827088 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:39.827289 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:39.827294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:39.828044 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:39.828045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:39.828856 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:39.828858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:39.829287 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:39.829289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:39.829415 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:39.829963 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:39.829972 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:39.831475 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:39.833005 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:39.833013 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:39.833016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:39.833266 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:39.833492 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:39.833493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:39.833493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:39.833606 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:39.833884 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:39.833891 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:39.834040 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 06:39:39.834145 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:39:39.834323 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260810 06:39:39.835208 18346 heartbeater.cc:347] Connected to a master server at 127.30.71.62:37099
I20260810 06:39:39.835312 18346 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:39.835489 18346 heartbeater.cc:511] Master 127.30.71.62:37099 requested a full tablet report, sending...
I20260810 06:39:39.836045 18140 ts_manager.cc:194] Registered new tserver with Master: 0a9e280b6c8b49e2a1104133a765d1df (127.30.71.1:45977)
I20260810 06:39:39.836736 18140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:58221
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:39:39.862969 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37099
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:39.949558 18351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:39.949782 18351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:39.949807 18351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:39.951027 18351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:39.951072 18351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:39.951098 18351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:39:39.952490 18351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:39.952713 18351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:39.952936 18351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:39.958202 18356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:39.958352 18351 server_base.cc:1034] running on GCE node
W20260810 06:39:39.958176 18357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:39.958169 18359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:39.958626 18351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:39.958837 18351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:39.959985 18351 hybrid_clock.cc:648] HybridClock initialized: now 1786343979959969 us; error 29 us; skew 500 ppm
Aug 10 06:39:39 dist-test-slave-w206 krb5kdc[18095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:39.962586 18351 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:39:39.963119 18351 webserver.cc:460] Webserver started at http://127.30.71.2:41323/ using document root <none> and password file <none>
I20260810 06:39:39.963320 18351 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:39.963362 18351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:39.963454 18351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:39.964498 18351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "e66fcf126d2a45789844a994a09127ce"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "7609ef50fcfad84860bdb4c7a9559ac2"
server_key_iv: "a7ef5b50623ef44f629d902fb846ce26"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.964797 18351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "e66fcf126d2a45789844a994a09127ce"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "7609ef50fcfad84860bdb4c7a9559ac2"
server_key_iv: "a7ef5b50623ef44f629d902fb846ce26"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.966063 18351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:39.966833 18366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:39.967051 18351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:39.967154 18351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "e66fcf126d2a45789844a994a09127ce"
format_stamp: "Formatted at 2026-08-10 06:39:39 on dist-test-slave-w206"
server_key: "7609ef50fcfad84860bdb4c7a9559ac2"
server_key_iv: "a7ef5b50623ef44f629d902fb846ce26"
server_key_version: "encryptionkey@0"
I20260810 06:39:39.967285 18351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:39.989624 18351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:39.991997 18351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:39.992182 18351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:39.992442 18351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:39.992779 18351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:39.992835 18351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:39.992882 18351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:39.992906 18351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:39.999240 18351 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:33979
I20260810 06:39:39.999310 18487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:33979 every 8 connection(s)
I20260810 06:39:39.999619 18351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
Aug 10 06:39:40 dist-test-slave-w206 krb5kdc[18095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343979, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:40.008849 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 18351
I20260810 06:39:40.008994 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:39:40.009208 31004 external_mini_cluster.cc:1428] Setting key 5c23c57ad6d0f2624a979eed837fb0e8
I20260810 06:39:40.009950 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.000371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:37937 (local address 127.30.71.62:37099)
0810 06:39:40.000454 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.000456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.000735 (+ 279us) server_negotiation.cc:407] Connection header received
0810 06:39:40.001236 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.001258 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.001269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.001331 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.002330 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.002842 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.003887 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.004104 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.004172 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.006521 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.006524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.006524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:40.006532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.008094 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.008696 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.008697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.008698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.008768 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.009091 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.009092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.009092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.009230 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.009291 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.009746 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.009855 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:39:40.010138 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.000313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37099 (local address 127.30.71.2:37937)
0810 06:39:40.000535 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.000629 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.001136 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.001415 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.001418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.002010 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:40.002219 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.002224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.002934 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.002934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.003799 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.003801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.004187 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.004188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.004292 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.004876 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:40.004885 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:40.006411 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:40.008280 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:40.008289 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:40.008292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:40.008574 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:40.008844 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:40.008845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:40.008845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:40.008994 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:40.009347 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:40.009356 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:40.009561 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.009699 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.009880 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
I20260810 06:39:40.010720 18488 heartbeater.cc:347] Connected to a master server at 127.30.71.62:37099
I20260810 06:39:40.010835 18488 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:40.011070 18488 heartbeater.cc:511] Master 127.30.71.62:37099 requested a full tablet report, sending...
I20260810 06:39:40.011590 18140 ts_manager.cc:194] Registered new tserver with Master: e66fcf126d2a45789844a994a09127ce (127.30.71.2:33979)
I20260810 06:39:40.012018 18140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:37937
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:39:40.039273 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37099
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:40.124122 18494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:40.124363 18494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:40.124392 18494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:40.125569 18494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:40.125617 18494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:40.125644 18494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:39:40.127142 18494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:37099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:40.127355 18494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:40.127557 18494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:40.132862 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.132903 18499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.132956 18494 server_base.cc:1034] running on GCE node
W20260810 06:39:40.132917 18502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.133313 18494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:40.133541 18494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:40.134694 18494 hybrid_clock.cc:648] HybridClock initialized: now 1786343980134674 us; error 44 us; skew 500 ppm
Aug 10 06:39:40 dist-test-slave-w206 krb5kdc[18095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343980, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:40.137488 18494 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:39:40.137908 18494 webserver.cc:460] Webserver started at http://127.30.71.3:39793/ using document root <none> and password file <none>
I20260810 06:39:40.138139 18494 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:40.138182 18494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:40.138314 18494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:40.139467 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "9728438b2fc44cf3bdea8a397ebe3352"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "cece946df8e5c73d7f40073abed640f2"
server_key_iv: "7c5c4e7a2dec1b2abef71fbb5f823f57"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.139772 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "9728438b2fc44cf3bdea8a397ebe3352"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "cece946df8e5c73d7f40073abed640f2"
server_key_iv: "7c5c4e7a2dec1b2abef71fbb5f823f57"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.141119 18494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:39:40.141925 18509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.142132 18494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:40.142220 18494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "9728438b2fc44cf3bdea8a397ebe3352"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "cece946df8e5c73d7f40073abed640f2"
server_key_iv: "7c5c4e7a2dec1b2abef71fbb5f823f57"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.142280 18494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:40.151292 18494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:40.153635 18494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:40.153837 18494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:40.154152 18494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:40.154552 18494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:40.154608 18494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.154633 18494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:40.154671 18494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.161566 18494 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:46399
I20260810 06:39:40.161731 18632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:46399 every 8 connection(s)
I20260810 06:39:40.162070 18494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:39:40.164881 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 18494
I20260810 06:39:40.164981 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:39:40.165185 31004 external_mini_cluster.cc:1428] Setting key e4e4be47d2cfed17556a2d1094fc6ad8
Aug 10 06:39:40 dist-test-slave-w206 krb5kdc[18095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343980, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:40.172860 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.162921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:57533 (local address 127.30.71.62:37099)
0810 06:39:40.163075 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.163078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.163284 (+ 206us) server_negotiation.cc:407] Connection header received
0810 06:39:40.163851 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.163852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.163866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.163927 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.165073 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.165892 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.167217 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.167421 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.167517 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.169955 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.169957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.169958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:40.169967 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.171390 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.171838 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.171839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.171839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.171902 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.172164 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.172165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.172165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.172298 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.172378 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.172671 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.172763 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260810 06:39:40.173067 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.162873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:37099 (local address 127.30.71.3:57533)
0810 06:39:40.163113 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.163173 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.163737 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.164011 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.164016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.164660 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:40.164925 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.164931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.166042 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.166042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.167127 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.167130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.167530 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.167532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.167677 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.168282 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:40.168291 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:40.169853 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:40.171499 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:40.171507 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:40.171509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:40.171758 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:40.171972 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:40.171972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:40.171973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:40.172096 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:40.172369 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:40.172376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:40.172530 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.172639 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.172822 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":122,"threads_started":1}
I20260810 06:39:40.173662 18633 heartbeater.cc:347] Connected to a master server at 127.30.71.62:37099
I20260810 06:39:40.173808 18633 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:40.174018 18633 heartbeater.cc:511] Master 127.30.71.62:37099 requested a full tablet report, sending...
I20260810 06:39:40.174492 18140 ts_manager.cc:194] Registered new tserver with Master: 9728438b2fc44cf3bdea8a397ebe3352 (127.30.71.3:46399)
I20260810 06:39:40.174885 18140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:57533
I20260810 06:39:40.177395 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:39:40.185609 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.178877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.30.71.62:37099)
0810 06:39:40.178927 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.178930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.179097 (+ 167us) server_negotiation.cc:407] Connection header received
0810 06:39:40.179152 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.179153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.179163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.179209 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.179797 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.180312 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.181055 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.181238 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.181353 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.182837 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.182839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.182839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:40.182846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.184347 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.184796 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.184797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.184798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.184856 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:40.185076 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:40.185077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:40.185077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:40.185228 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.185348 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.185424 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.185481 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:39:40.187557 18140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:39:40.189334 18140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:39:40.196593 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.192182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48716 (local address 127.30.71.1:45977)
0810 06:39:40.192394 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.192398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.192560 (+ 162us) server_negotiation.cc:407] Connection header received
0810 06:39:40.192844 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.192848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.192866 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.192930 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.193202 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.194298 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.195392 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.196139 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.196214 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.196281 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.196442 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.196485 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260810 06:39:40.196583 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.192026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46399 (local address 127.0.0.1:49946)
0810 06:39:40.192427 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.192470 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.192543 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.192874 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.192876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.192879 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.192974 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.192977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.194062 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.194064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.195447 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.195450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.196355 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.196356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.196424 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.196427 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.196476 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.196509 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":26,"thread_start_us":75,"threads_started":1}
I20260810 06:39:40.196583 18647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.192072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:45977 (local address 127.0.0.1:48716)
0810 06:39:40.192454 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.192489 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.192725 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.193065 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.193066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.193068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.193116 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.193118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.194440 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.194441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.195293 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.195296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.196235 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.196236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.196320 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.196335 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.196416 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.196483 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":103,"threads_started":1}
I20260810 06:39:40.196806 18646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.192288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:33979 (local address 127.0.0.1:53138)
0810 06:39:40.192473 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.192482 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.192543 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.192883 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.192884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.192885 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.192974 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.192977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.194413 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.194414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.195649 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.195651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.196568 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.196569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.196636 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.196641 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.196674 (+ 33us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.196721 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":66,"threads_started":1}
I20260810 06:39:40.196880 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.192122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.30.71.3:46399)
0810 06:39:40.192333 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.192340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.192638 (+ 298us) server_negotiation.cc:407] Connection header received
0810 06:39:40.192688 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.192691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.192709 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.192783 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.193066 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.193943 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.195551 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.196152 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.196257 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.196320 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.196725 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.196781 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":111,"threads_started":1}
I20260810 06:39:40.197395 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.192488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.30.71.2:33979)
0810 06:39:40.192704 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.192708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.192717 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:39:40.192771 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.192774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.192787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.193056 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.193127 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.194298 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.195758 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.196470 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.197188 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.197250 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.197277 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.197317 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260810 06:39:40.197644 18275 tablet_service.cc:1467] Processing CreateTablet for tablet 26a71a1c3b0d414da6f4eb71c3209c14 (DEFAULT_TABLE table=test-table [id=de6de6e88f94439a9d019da7e9ea5ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:40.198031 18275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26a71a1c3b0d414da6f4eb71c3209c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:40.197995 18560 tablet_service.cc:1467] Processing CreateTablet for tablet 26a71a1c3b0d414da6f4eb71c3209c14 (DEFAULT_TABLE table=test-table [id=de6de6e88f94439a9d019da7e9ea5ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:40.198310 18560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26a71a1c3b0d414da6f4eb71c3209c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:40.198225 18417 tablet_service.cc:1467] Processing CreateTablet for tablet 26a71a1c3b0d414da6f4eb71c3209c14 (DEFAULT_TABLE table=test-table [id=de6de6e88f94439a9d019da7e9ea5ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:40.198570 18417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26a71a1c3b0d414da6f4eb71c3209c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:40.201465 18652 tablet_bootstrap.cc:492] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Bootstrap starting.
I20260810 06:39:40.201675 18653 tablet_bootstrap.cc:492] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Bootstrap starting.
I20260810 06:39:40.202039 18654 tablet_bootstrap.cc:492] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df: Bootstrap starting.
I20260810 06:39:40.202322 18653 tablet_bootstrap.cc:654] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:40.202391 18652 tablet_bootstrap.cc:654] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:40.202766 18654 tablet_bootstrap.cc:654] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:40.202889 18653 log.cc:824] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:40.203145 18652 log.cc:824] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:40.203457 18654 log.cc:824] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:40.203761 18653 tablet_bootstrap.cc:492] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: No bootstrap required, opened a new log
I20260810 06:39:40.203840 18653 ts_tablet_manager.cc:1397] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:39:40.204316 18654 tablet_bootstrap.cc:492] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df: No bootstrap required, opened a new log
I20260810 06:39:40.204393 18654 ts_tablet_manager.cc:1397] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:40.205524 18653 raft_consensus.cc:348] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.205690 18653 raft_consensus.cc:374] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:40.205739 18653 raft_consensus.cc:729] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9728438b2fc44cf3bdea8a397ebe3352, State: Initialized, Role: FOLLOWER
I20260810 06:39:40.205863 18653 consensus_queue.cc:260] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.206156 18653 ts_tablet_manager.cc:1428] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 06:39:40.206197 18633 heartbeater.cc:503] Master 127.30.71.62:37099 was elected leader, sending a full tablet report...
I20260810 06:39:40.206259 18654 raft_consensus.cc:348] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.206442 18654 raft_consensus.cc:374] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:40.206597 18654 raft_consensus.cc:729] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a9e280b6c8b49e2a1104133a765d1df, State: Initialized, Role: FOLLOWER
I20260810 06:39:40.206727 18654 consensus_queue.cc:260] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.206773 18652 tablet_bootstrap.cc:492] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: No bootstrap required, opened a new log
I20260810 06:39:40.206840 18652 ts_tablet_manager.cc:1397] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:39:40.207049 18654 ts_tablet_manager.cc:1428] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:40.207237 18346 heartbeater.cc:503] Master 127.30.71.62:37099 was elected leader, sending a full tablet report...
I20260810 06:39:40.208773 18652 raft_consensus.cc:348] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.208901 18652 raft_consensus.cc:374] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:40.208922 18652 raft_consensus.cc:729] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e66fcf126d2a45789844a994a09127ce, State: Initialized, Role: FOLLOWER
I20260810 06:39:40.209025 18652 consensus_queue.cc:260] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.209321 18488 heartbeater.cc:503] Master 127.30.71.62:37099 was elected leader, sending a full tablet report...
I20260810 06:39:40.209306 18652 ts_tablet_manager.cc:1428] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260810 06:39:40.250176 18489 tablet.cc:2369] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:40.251320 18660 raft_consensus.cc:482] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:40.251466 18660 raft_consensus.cc:504] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.251864 18660 leader_election.cc:288] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a9e280b6c8b49e2a1104133a765d1df (127.30.71.1:45977), 9728438b2fc44cf3bdea8a397ebe3352 (127.30.71.3:46399)
I20260810 06:39:40.255288 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.252140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:45977 (local address 127.30.71.2:34059)
0810 06:39:40.252208 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.252224 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.252318 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.252518 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.252519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.252520 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.252604 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.252606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.253329 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.253330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.254204 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.254207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.255023 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.255024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.255109 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.255112 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.255157 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.255194 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 06:39:40.255298 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.252315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:34059 (local address 127.30.71.1:45977)
0810 06:39:40.252391 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.252393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.252404 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:39:40.252434 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.252435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.252446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.252513 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.252684 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.253230 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.254305 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.254911 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.255019 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.255031 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.255180 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.255220 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:39:40.255617 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.252140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:46399 (local address 127.30.71.2:45717)
0810 06:39:40.252460 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.252471 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.252624 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.252819 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.252820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.252821 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.252899 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.252901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.253567 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.253568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.254504 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.254507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.255281 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.255282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.255351 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.255356 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.255512 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.255551 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":65,"thread_start_us":125,"threads_started":1}
I20260810 06:39:40.255643 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.252245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:45717 (local address 127.30.71.3:46399)
0810 06:39:40.252289 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.252291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.252602 (+ 311us) server_negotiation.cc:407] Connection header received
0810 06:39:40.252711 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.252713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.252750 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.252816 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.252979 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.253478 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.254594 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.255201 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.255272 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.255288 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.255534 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.255576 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:39:40.255636 18297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26a71a1c3b0d414da6f4eb71c3209c14" candidate_uuid: "e66fcf126d2a45789844a994a09127ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a9e280b6c8b49e2a1104133a765d1df" is_pre_election: true
I20260810 06:39:40.255811 18297 raft_consensus.cc:2454] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e66fcf126d2a45789844a994a09127ce in term 0.
I20260810 06:39:40.256086 18372 leader_election.cc:302] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a9e280b6c8b49e2a1104133a765d1df, e66fcf126d2a45789844a994a09127ce; no voters:
I20260810 06:39:40.256165 18582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26a71a1c3b0d414da6f4eb71c3209c14" candidate_uuid: "e66fcf126d2a45789844a994a09127ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9728438b2fc44cf3bdea8a397ebe3352" is_pre_election: true
I20260810 06:39:40.256208 18660 raft_consensus.cc:2781] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:39:40.256264 18660 raft_consensus.cc:482] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:39:40.256316 18660 raft_consensus.cc:3037] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:40.256325 18582 raft_consensus.cc:2454] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e66fcf126d2a45789844a994a09127ce in term 0.
I20260810 06:39:40.257124 18660 raft_consensus.cc:504] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.257339 18660 leader_election.cc:288] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [CANDIDATE]: Term 1 election: Requested vote from peers 0a9e280b6c8b49e2a1104133a765d1df (127.30.71.1:45977), 9728438b2fc44cf3bdea8a397ebe3352 (127.30.71.3:46399)
I20260810 06:39:40.257426 18297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26a71a1c3b0d414da6f4eb71c3209c14" candidate_uuid: "e66fcf126d2a45789844a994a09127ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a9e280b6c8b49e2a1104133a765d1df"
I20260810 06:39:40.257489 18582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26a71a1c3b0d414da6f4eb71c3209c14" candidate_uuid: "e66fcf126d2a45789844a994a09127ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9728438b2fc44cf3bdea8a397ebe3352"
I20260810 06:39:40.257513 18297 raft_consensus.cc:3037] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:40.257560 18582 raft_consensus.cc:3037] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:40.258499 18297 raft_consensus.cc:2454] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e66fcf126d2a45789844a994a09127ce in term 1.
I20260810 06:39:40.258499 18582 raft_consensus.cc:2454] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e66fcf126d2a45789844a994a09127ce in term 1.
I20260810 06:39:40.258673 18372 leader_election.cc:302] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a9e280b6c8b49e2a1104133a765d1df, e66fcf126d2a45789844a994a09127ce; no voters:
I20260810 06:39:40.258773 18660 raft_consensus.cc:2781] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:40.258929 18660 raft_consensus.cc:686] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 1 LEADER]: Becoming Leader. State: Replica: e66fcf126d2a45789844a994a09127ce, State: Running, Role: LEADER
I20260810 06:39:40.259050 18660 consensus_queue.cc:237] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } }
I20260810 06:39:40.259924 18140 catalog_manager.cc:5495] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce reported cstate change: term changed from 0 to 1, leader changed from <none> to e66fcf126d2a45789844a994a09127ce (127.30.71.2). New cstate: current_term: 1 leader_uuid: "e66fcf126d2a45789844a994a09127ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e66fcf126d2a45789844a994a09127ce" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 33979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 } health_report { overall_health: UNKNOWN } } }
I20260810 06:39:40.274039 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.271903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.30.71.2:33979)
0810 06:39:40.271943 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.271947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.271960 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:39:40.272016 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.272016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.272034 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.272095 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:39:40.272212 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.272671 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.273403 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.273578 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.273627 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.273753 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:39:40.273854 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:39:40.273923 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.273972 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.273980 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:39:40.275982 18582 raft_consensus.cc:1261] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 1 FOLLOWER]: Refusing update from remote peer e66fcf126d2a45789844a994a09127ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:40.275995 18297 raft_consensus.cc:1261] T 26a71a1c3b0d414da6f4eb71c3209c14 P 0a9e280b6c8b49e2a1104133a765d1df [term 1 FOLLOWER]: Refusing update from remote peer e66fcf126d2a45789844a994a09127ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:40.276376 18660 consensus_queue.cc:1035] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "9728438b2fc44cf3bdea8a397ebe3352" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 46399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:40.276527 18662 consensus_queue.cc:1035] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a9e280b6c8b49e2a1104133a765d1df" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:40.279695 18667 mvcc.cc:205] Tried to move back new op lower bound from 7316864943207247872 to 7316864943141773312. Current Snapshot: MvccSnapshot[applied={T|T < 7316864943207247872 or (T in {7316864943207247872})}]
I20260810 06:39:40.280082 18668 mvcc.cc:205] Tried to move back new op lower bound from 7316864943207247872 to 7316864943141773312. Current Snapshot: MvccSnapshot[applied={T|T < 7316864943207247872}]
I20260810 06:39:40.281355 18140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:40.281461 18140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:40.282222 18140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 190931ede35842a6a9efd387caa02354: Sending DeleteTablet for 3 replicas of tablet 26a71a1c3b0d414da6f4eb71c3209c14
I20260810 06:39:40.282595 18560 tablet_service.cc:1514] Processing DeleteTablet for tablet 26a71a1c3b0d414da6f4eb71c3209c14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:40 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:49946
I20260810 06:39:40.282698 18417 tablet_service.cc:1514] Processing DeleteTablet for tablet 26a71a1c3b0d414da6f4eb71c3209c14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:40 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:53138
I20260810 06:39:40.282958 18678 tablet_replica.cc:330] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: stopping tablet replica
I20260810 06:39:40.283005 18275 tablet_service.cc:1514] Processing DeleteTablet for tablet 26a71a1c3b0d414da6f4eb71c3209c14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:40 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:48716
I20260810 06:39:40.283063 18677 tablet_replica.cc:330] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: stopping tablet replica
I20260810 06:39:40.283170 18677 raft_consensus.cc:2229] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:40.283282 18677 raft_consensus.cc:2258] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:40.283275 18678 raft_consensus.cc:2229] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:39:40.283466 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 18209
I20260810 06:39:40.285408 18677 ts_tablet_manager.cc:1905] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:40.287880 18678 raft_consensus.cc:2258] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:40.288237 18677 ts_tablet_manager.cc:1918] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:40.288358 18677 log.cc:1196] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/26a71a1c3b0d414da6f4eb71c3209c14
I20260810 06:39:40.288659 18677 ts_tablet_manager.cc:1939] T 26a71a1c3b0d414da6f4eb71c3209c14 P 9728438b2fc44cf3bdea8a397ebe3352: Deleting consensus metadata
I20260810 06:39:40.289175 18126 catalog_manager.cc:4841] TS 9728438b2fc44cf3bdea8a397ebe3352 (127.30.71.3:46399): tablet 26a71a1c3b0d414da6f4eb71c3209c14 (table test-table [id=de6de6e88f94439a9d019da7e9ea5ec4]) successfully deleted
I20260810 06:39:40.289332 18678 ts_tablet_manager.cc:1905] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:40.291152 18678 ts_tablet_manager.cc:1918] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:40.291244 18678 log.cc:1196] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/26a71a1c3b0d414da6f4eb71c3209c14
I20260810 06:39:40.291555 18678 ts_tablet_manager.cc:1939] T 26a71a1c3b0d414da6f4eb71c3209c14 P e66fcf126d2a45789844a994a09127ce: Deleting consensus metadata
W20260810 06:39:40.291867 18127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:45977 (error 108)
I20260810 06:39:40.292147 18127 catalog_manager.cc:4841] TS e66fcf126d2a45789844a994a09127ce (127.30.71.2:33979): tablet 26a71a1c3b0d414da6f4eb71c3209c14 (table test-table [id=de6de6e88f94439a9d019da7e9ea5ec4]) successfully deleted
W20260810 06:39:40.292529 18646 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:39:40.292342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:45977 (local address 127.0.0.1:48720)
0810 06:39:40.292405 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.292433 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.292478 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:45977: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:39:40.292657 18127 catalog_manager.cc:4584] TS 0a9e280b6c8b49e2a1104133a765d1df (127.30.71.1:45977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26a71a1c3b0d414da6f4eb71c3209c14: Network error: Client connection negotiation failed: client connection to 127.30.71.1:45977: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260810 06:39:40.292809 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 18351
I20260810 06:39:40.298544 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 18494
I20260810 06:39:40.304690 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 18108
2026-08-10T06:39:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2007 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T06:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:40Z Disabled control of system clock
I20260810 06:39:40.335348 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42647
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:35403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:42647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:40.418785 18684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:40.419068 18684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:40.419092 18684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:40.420437 18684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:40.420486 18684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:40.420499 18684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:40.420512 18684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:40.420565 18684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:40.421949 18684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:42647
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:40.422224 18684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:40.422554 18684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:39:40.427681 18684 server_base.cc:1034] running on GCE node
W20260810 06:39:40.427625 18689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.427700 18690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.427634 18692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.428081 18684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:40.428378 18684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:40.429556 18684 hybrid_clock.cc:648] HybridClock initialized: now 1786343980429523 us; error 48 us; skew 500 ppm
I20260810 06:39:40.436462 18684 webserver.cc:460] Webserver started at http://127.30.71.62:46363/ using document root <none> and password file <none>
I20260810 06:39:40.436764 18684 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:40.436867 18684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:40.437022 18684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:40.438153 18684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "3d55a5c76907434daa7a3e83b49a916d"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "cdc290b3c7a0887470c132ec5daba1fb"
server_key_iv: "fb5b70c060feb17e7cc15766f26b161f"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.438519 18684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "3d55a5c76907434daa7a3e83b49a916d"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "cdc290b3c7a0887470c132ec5daba1fb"
server_key_iv: "fb5b70c060feb17e7cc15766f26b161f"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.440038 18684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:40.440840 18699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.441030 18684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:40.441139 18684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "3d55a5c76907434daa7a3e83b49a916d"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "cdc290b3c7a0887470c132ec5daba1fb"
server_key_iv: "fb5b70c060feb17e7cc15766f26b161f"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.441212 18684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:40.448534 18684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:40.448841 18684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:40.448983 18684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:40.453024 18684 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:42647
I20260810 06:39:40.453065 18760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:42647 every 8 connection(s)
I20260810 06:39:40.453414 18684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:40.454563 18761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:40.457274 18761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d: Bootstrap starting.
I20260810 06:39:40.457973 18761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:40.458302 18761 log.cc:824] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:40.459075 18761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d: No bootstrap required, opened a new log
I20260810 06:39:40.460181 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 18684
I20260810 06:39:40.460330 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:40.460532 31004 external_mini_cluster.cc:1428] Setting key e7e8ba99ed8aa25e5aeb18c677818bd1
I20260810 06:39:40.461191 18761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d55a5c76907434daa7a3e83b49a916d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42647 } }
I20260810 06:39:40.461421 18761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:40.461480 18761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d55a5c76907434daa7a3e83b49a916d, State: Initialized, Role: FOLLOWER
I20260810 06:39:40.461620 18761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d55a5c76907434daa7a3e83b49a916d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42647 } }
I20260810 06:39:40.461719 18761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:40.461757 18761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:40.461812 18761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:40.462423 18761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d55a5c76907434daa7a3e83b49a916d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42647 } }
I20260810 06:39:40.462517 18761 leader_election.cc:302] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d55a5c76907434daa7a3e83b49a916d; no voters:
I20260810 06:39:40.462720 18761 leader_election.cc:288] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:40.462769 18766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:40.462896 18766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [term 1 LEADER]: Becoming Leader. State: Replica: 3d55a5c76907434daa7a3e83b49a916d, State: Running, Role: LEADER
I20260810 06:39:40.463042 18761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:40.463011 18766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d55a5c76907434daa7a3e83b49a916d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42647 } }
I20260810 06:39:40.463402 18767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d55a5c76907434daa7a3e83b49a916d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d55a5c76907434daa7a3e83b49a916d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42647 } } }
I20260810 06:39:40.463495 18767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:40.463680 18768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d55a5c76907434daa7a3e83b49a916d. Latest consensus state: current_term: 1 leader_uuid: "3d55a5c76907434daa7a3e83b49a916d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d55a5c76907434daa7a3e83b49a916d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42647 } } }
I20260810 06:39:40.463752 18768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d [sys.catalog]: This master's current role is: LEADER
W20260810 06:39:40.464986 18782 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:39:40.465052 18782 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:39:40.465111 18771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:40.465284 18771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:40.467108 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.461132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.30.71.62:42647)
0810 06:39:40.461345 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.461352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.461377 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:39:40.461499 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.461506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.461577 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.461697 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.461954 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.463494 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.464557 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.465136 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.465392 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.466226 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.466230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.466230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:40.466439 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.466484 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:40.466499 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.466563 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.466771 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.466880 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260810 06:39:40.467123 18771 catalog_manager.cc:1347] Generated new cluster ID: 5ed020a3581d463b9568a764c418852d
I20260810 06:39:40.467295 18771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:40.500708 18771 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:40.501279 18771 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:40.506141 18771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d: Generated new TSK 0
I20260810 06:39:40.506348 18771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:40.513613 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42647
--builtin_ntp_servers=127.30.71.20:35403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:40.597735 18786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:40.597947 18786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:40.597970 18786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:40.599295 18786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:40.599337 18786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:40.599364 18786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:39:40.600723 18786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:40.600914 18786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:40.601176 18786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:40.606094 18791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.606129 18792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.606124 18794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.606523 18786 server_base.cc:1034] running on GCE node
I20260810 06:39:40.606767 18786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:40.607070 18786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:40.608245 18786 hybrid_clock.cc:648] HybridClock initialized: now 1786343980608228 us; error 28 us; skew 500 ppm
I20260810 06:39:40.609586 18786 webserver.cc:460] Webserver started at http://127.30.71.1:33019/ using document root <none> and password file <none>
I20260810 06:39:40.609838 18786 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:40.609906 18786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:40.610059 18786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:40.611217 18786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "b62a7bc5d5e846409cad5b95bc60df9c"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "e2c4d06e5df1953d7fa0437d43b73b24"
server_key_iv: "643a42282f8f71af32ff9a39fdc9633f"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.611577 18786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "b62a7bc5d5e846409cad5b95bc60df9c"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "e2c4d06e5df1953d7fa0437d43b73b24"
server_key_iv: "643a42282f8f71af32ff9a39fdc9633f"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.612905 18786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:39:40.613754 18800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.613955 18786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:40.614059 18786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "b62a7bc5d5e846409cad5b95bc60df9c"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "e2c4d06e5df1953d7fa0437d43b73b24"
server_key_iv: "643a42282f8f71af32ff9a39fdc9633f"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.614142 18786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:40.630716 18786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:40.631125 18786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:40.631289 18786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:40.631538 18786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:40.631924 18786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:40.631981 18786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.632025 18786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:40.632051 18786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.637868 18786 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:46599
I20260810 06:39:40.637913 18921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:46599 every 8 connection(s)
I20260810 06:39:40.638263 18786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:39:40.639107 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 18786
I20260810 06:39:40.639246 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:39:40.639465 31004 external_mini_cluster.cc:1428] Setting key c8eefa4477dbbf17558a6957699d110e
I20260810 06:39:40.640596 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42647
--builtin_ntp_servers=127.30.71.20:35403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:39:40.644467 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.639055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:43055 (local address 127.30.71.62:42647)
0810 06:39:40.639117 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.639119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.639477 (+ 358us) server_negotiation.cc:407] Connection header received
0810 06:39:40.640236 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.640237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.640271 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.640320 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.641111 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.641878 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.642867 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.643161 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.643249 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.644225 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.644227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.644227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:40.644233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.644249 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:40.644253 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.644295 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.644379 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.644407 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:39:40.644774 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.639059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42647 (local address 127.30.71.1:43055)
0810 06:39:40.639338 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.639389 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.640106 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.640401 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.640405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.640414 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:40.641017 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.641022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.641973 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.641973 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.642767 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.642769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.643261 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.643262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.643466 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.644031 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:40.644031 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:40.644071 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:40.644102 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:40.644305 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:40.644311 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:40.644314 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.644383 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.644497 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":97,"threads_started":1}
I20260810 06:39:40.645315 18922 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42647
I20260810 06:39:40.645437 18922 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:40.645661 18922 heartbeater.cc:511] Master 127.30.71.62:42647 requested a full tablet report, sending...
I20260810 06:39:40.646195 18718 ts_manager.cc:194] Registered new tserver with Master: b62a7bc5d5e846409cad5b95bc60df9c (127.30.71.1:46599)
I20260810 06:39:40.646834 18718 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.1:43055
W20260810 06:39:40.726094 18926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:40.726336 18926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:40.726373 18926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:40.727597 18926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:40.727677 18926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:40.727717 18926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:39:40.729101 18926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:40.729385 18926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:40.729697 18926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:40.734771 18932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.734918 18926 server_base.cc:1034] running on GCE node
W20260810 06:39:40.734781 18934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.734794 18931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.735340 18926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:40.735544 18926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:40.736677 18926 hybrid_clock.cc:648] HybridClock initialized: now 1786343980736668 us; error 28 us; skew 500 ppm
I20260810 06:39:40.737947 18926 webserver.cc:460] Webserver started at http://127.30.71.2:33623/ using document root <none> and password file <none>
I20260810 06:39:40.738247 18926 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:40.738300 18926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:40.738469 18926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:40.739638 18926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "87e0dffcb6874a55982e49b136bcaf57"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "7d44fc1369e2465dca778e48034684a6"
server_key_iv: "e012d50e74e69679c99f0518c7389d79"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.740015 18926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "87e0dffcb6874a55982e49b136bcaf57"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "7d44fc1369e2465dca778e48034684a6"
server_key_iv: "e012d50e74e69679c99f0518c7389d79"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.741338 18926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 06:39:40.742079 18940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.742316 18926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:40.742419 18926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "87e0dffcb6874a55982e49b136bcaf57"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "7d44fc1369e2465dca778e48034684a6"
server_key_iv: "e012d50e74e69679c99f0518c7389d79"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.742493 18926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:40.753177 18926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:40.753489 18926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:40.753638 18926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:40.753860 18926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:40.754195 18926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:40.754251 18926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.754298 18926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:40.754343 18926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.760372 18926 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:39107
I20260810 06:39:40.760413 19060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:39107 every 8 connection(s)
I20260810 06:39:40.760759 18926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:39:40.765918 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.761542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:44349 (local address 127.30.71.62:42647)
0810 06:39:40.761633 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.761635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.761820 (+ 185us) server_negotiation.cc:407] Connection header received
0810 06:39:40.762297 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.762298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.762308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.762362 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.763123 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.763661 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.764701 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.764886 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.764944 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.765721 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.765723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.765723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:40.765727 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.765741 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:40.765744 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.765786 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.765836 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.765859 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:39:40.766110 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.761500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42647 (local address 127.30.71.2:44349)
0810 06:39:40.761697 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.761731 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.762203 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.762469 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.762473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.762480 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:40.763048 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.763053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.763769 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.763774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.764630 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.764632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.764965 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.764966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.765082 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.765595 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:40.765595 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:40.765623 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:40.765645 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:40.765788 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:40.765790 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:40.765792 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.765837 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.765910 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260810 06:39:40.766606 19061 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42647
I20260810 06:39:40.766716 19061 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:40.766906 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 18926
I20260810 06:39:40.766923 19061 heartbeater.cc:511] Master 127.30.71.62:42647 requested a full tablet report, sending...
I20260810 06:39:40.767040 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:39:40.767289 31004 external_mini_cluster.cc:1428] Setting key 576ed63943c86c77e05da462296cae8c
I20260810 06:39:40.767397 18718 ts_manager.cc:194] Registered new tserver with Master: 87e0dffcb6874a55982e49b136bcaf57 (127.30.71.2:39107)
I20260810 06:39:40.767793 18718 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.2:44349
I20260810 06:39:40.768381 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42647
--builtin_ntp_servers=127.30.71.20:35403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:40.852252 19064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:40.852455 19064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:40.852478 19064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:40.853579 19064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:40.853614 19064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:40.853638 19064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:39:40.854933 19064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:42647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:40.855171 19064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:40.855508 19064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:40.860529 19073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.860538 19071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:40.860605 19070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:40.860661 19064 server_base.cc:1034] running on GCE node
I20260810 06:39:40.860942 19064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:40.861181 19064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:40.862358 19064 hybrid_clock.cc:648] HybridClock initialized: now 1786343980862341 us; error 28 us; skew 500 ppm
I20260810 06:39:40.863756 19064 webserver.cc:460] Webserver started at http://127.30.71.3:44997/ using document root <none> and password file <none>
I20260810 06:39:40.864094 19064 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:40.864164 19064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:40.864298 19064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:40.865393 19064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "03c897fec3c449eeb351a303d67fe80b"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "8eb08ddeaf2f530679b4ab9147e21ad6"
server_key_iv: "9e726c7db865cc810d9d8590295c39d5"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.865754 19064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "03c897fec3c449eeb351a303d67fe80b"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "8eb08ddeaf2f530679b4ab9147e21ad6"
server_key_iv: "9e726c7db865cc810d9d8590295c39d5"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.867085 19064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:40.867817 19079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.867995 19064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:39:40.868089 19064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "03c897fec3c449eeb351a303d67fe80b"
format_stamp: "Formatted at 2026-08-10 06:39:40 on dist-test-slave-w206"
server_key: "8eb08ddeaf2f530679b4ab9147e21ad6"
server_key_iv: "9e726c7db865cc810d9d8590295c39d5"
server_key_version: "encryptionkey@0"
I20260810 06:39:40.868173 19064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:40.876063 19064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:40.876353 19064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:40.876499 19064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:40.876713 19064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:40.877030 19064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:40.877084 19064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.877118 19064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:40.877148 19064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:40.883473 19064 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:38297
I20260810 06:39:40.883541 19198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:38297 every 8 connection(s)
I20260810 06:39:40.883896 19064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:39:40.888957 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.884661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:34341 (local address 127.30.71.62:42647)
0810 06:39:40.884746 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.884748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.884982 (+ 234us) server_negotiation.cc:407] Connection header received
0810 06:39:40.885498 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.885499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.885509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.885562 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.886205 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.886697 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.887726 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.887916 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.887972 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.888734 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.888735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.888735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:40.888739 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.888753 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:40.888757 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.888800 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.888871 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.888896 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:39:40.889168 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.884638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:42647 (local address 127.30.71.3:34341)
0810 06:39:40.884854 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.884889 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.885408 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.885613 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.885616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.885624 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:40.886127 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.886132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.886778 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.886779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.887647 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.887650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.887990 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.887991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.888094 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.888590 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:40.888590 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:40.888618 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:40.888640 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:40.888800 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:40.888805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:40.888807 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.888855 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.888932 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
I20260810 06:39:40.889744 19199 heartbeater.cc:347] Connected to a master server at 127.30.71.62:42647
I20260810 06:39:40.889849 19199 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:40.890031 19199 heartbeater.cc:511] Master 127.30.71.62:42647 requested a full tablet report, sending...
I20260810 06:39:40.890398 18718 ts_manager.cc:194] Registered new tserver with Master: 03c897fec3c449eeb351a303d67fe80b (127.30.71.3:38297)
I20260810 06:39:40.890760 18718 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.3:34341
I20260810 06:39:40.893707 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 19064
I20260810 06:39:40.893841 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:39:40.894058 31004 external_mini_cluster.cc:1428] Setting key a49aa7f48505792c539e81bb6dc830fc
I20260810 06:39:40.894927 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:39:40.899096 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.896360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.30.71.62:42647)
0810 06:39:40.896420 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.896422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.896486 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:39:40.896545 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.896546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.896555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.896602 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:40.896754 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.897206 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.897934 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.898105 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.898153 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.898826 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:40.898827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:40.898827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:40.898831 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:40.898842 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:40.898846 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:40.898907 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.899023 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.899033 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:39:40.900835 18718 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:39:40.902328 18718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:39:40.908739 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.904458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:39107 (local address 127.0.0.1:50884)
0810 06:39:40.904613 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.904656 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.904725 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.905109 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.905112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.905114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.905222 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.905223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.906600 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.906601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.907528 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.907530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.908447 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.908448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.908521 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.908533 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.908612 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.908663 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":56,"thread_start_us":61,"threads_started":1}
I20260810 06:39:40.908752 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.904626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50884 (local address 127.30.71.2:39107)
0810 06:39:40.904936 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.904940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.904950 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:39:40.904989 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.904991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.905007 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.905096 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.905354 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.906536 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.907632 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.908363 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.908448 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.908555 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.908626 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.908663 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":75,"threads_started":1}
I20260810 06:39:40.909615 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.904467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:46599 (local address 127.0.0.1:39950)
0810 06:39:40.904781 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.904791 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.904843 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.905109 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.905112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.905117 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.905212 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.905214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.906558 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.906560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.908132 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.908135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.909377 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.909378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.909440 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.909444 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.909510 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.909551 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":61,"thread_start_us":89,"threads_started":1}
I20260810 06:39:40.909466 18989 tablet_service.cc:1467] Processing CreateTablet for tablet 10eb07922691415f91c828b768144fee (DEFAULT_TABLE table=test-table [id=6048cf48d07c48168675ffb4547100ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:40.909652 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.904579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.30.71.1:46599)
0810 06:39:40.904831 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.904837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.904849 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:39:40.904941 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.904944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.904962 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.905034 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.905313 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.906469 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.908235 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.909269 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.909363 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.909444 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.909520 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.909560 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":103,"threads_started":1}
I20260810 06:39:40.909883 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.904658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38297 (local address 127.0.0.1:34034)
0810 06:39:40.905195 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.905206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.905262 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.906257 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.906258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.906260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.906313 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.906315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.907584 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.907585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.908365 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.908367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.909481 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.909482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.909542 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.909546 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.909595 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.909633 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":67,"threads_started":1}
I20260810 06:39:40.909919 18989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10eb07922691415f91c828b768144fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:40.910156 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.904880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.30.71.3:38297)
0810 06:39:40.906102 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.906106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.906117 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:39:40.906160 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.906161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.906173 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.906260 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.906371 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.907500 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.908461 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.909304 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.909363 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.909444 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.910018 (+ 574us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.910069 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1148,"thread_start_us":89,"threads_started":1}
I20260810 06:39:40.910501 18850 tablet_service.cc:1467] Processing CreateTablet for tablet 10eb07922691415f91c828b768144fee (DEFAULT_TABLE table=test-table [id=6048cf48d07c48168675ffb4547100ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:40.910810 18850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10eb07922691415f91c828b768144fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:40.910748 19128 tablet_service.cc:1467] Processing CreateTablet for tablet 10eb07922691415f91c828b768144fee (DEFAULT_TABLE table=test-table [id=6048cf48d07c48168675ffb4547100ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:40.911104 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10eb07922691415f91c828b768144fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:40.913407 19214 tablet_bootstrap.cc:492] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c: Bootstrap starting.
I20260810 06:39:40.913795 19215 tablet_bootstrap.cc:492] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Bootstrap starting.
I20260810 06:39:40.914458 19215 tablet_bootstrap.cc:654] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:40.914458 19214 tablet_bootstrap.cc:654] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:40.914803 19214 log.cc:824] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:40.914796 19215 log.cc:824] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:40.915143 19216 tablet_bootstrap.cc:492] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Bootstrap starting.
I20260810 06:39:40.915544 19214 tablet_bootstrap.cc:492] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c: No bootstrap required, opened a new log
I20260810 06:39:40.915604 19215 tablet_bootstrap.cc:492] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: No bootstrap required, opened a new log
I20260810 06:39:40.915628 19214 ts_tablet_manager.cc:1397] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 06:39:40.915673 19215 ts_tablet_manager.cc:1397] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:39:40.915791 19216 tablet_bootstrap.cc:654] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:40.916093 19216 log.cc:824] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:40.916863 19216 tablet_bootstrap.cc:492] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: No bootstrap required, opened a new log
I20260810 06:39:40.916930 19216 ts_tablet_manager.cc:1397] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:39:40.918298 19215 raft_consensus.cc:348] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.918319 19214 raft_consensus.cc:348] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.918457 19215 raft_consensus.cc:374] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:40.918467 19214 raft_consensus.cc:374] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:40.918501 19215 raft_consensus.cc:729] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e0dffcb6874a55982e49b136bcaf57, State: Initialized, Role: FOLLOWER
I20260810 06:39:40.918501 19214 raft_consensus.cc:729] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b62a7bc5d5e846409cad5b95bc60df9c, State: Initialized, Role: FOLLOWER
I20260810 06:39:40.918617 19215 consensus_queue.cc:260] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.918663 19214 consensus_queue.cc:260] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.918893 19215 ts_tablet_manager.cc:1428] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:39:40.918934 19214 ts_tablet_manager.cc:1428] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:39:40.919047 19061 heartbeater.cc:503] Master 127.30.71.62:42647 was elected leader, sending a full tablet report...
I20260810 06:39:40.919083 18922 heartbeater.cc:503] Master 127.30.71.62:42647 was elected leader, sending a full tablet report...
I20260810 06:39:40.919641 19216 raft_consensus.cc:348] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.919796 19216 raft_consensus.cc:374] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:40.919842 19216 raft_consensus.cc:729] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c897fec3c449eeb351a303d67fe80b, State: Initialized, Role: FOLLOWER
I20260810 06:39:40.919998 19216 consensus_queue.cc:260] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.920269 19216 ts_tablet_manager.cc:1428] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:40.920323 19199 heartbeater.cc:503] Master 127.30.71.62:42647 was elected leader, sending a full tablet report...
I20260810 06:39:40.953989 19221 raft_consensus.cc:482] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:40.954145 19221 raft_consensus.cc:504] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.954541 19221 leader_election.cc:288] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87e0dffcb6874a55982e49b136bcaf57 (127.30.71.2:39107), 03c897fec3c449eeb351a303d67fe80b (127.30.71.3:38297)
I20260810 06:39:40.957577 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.954786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:52473 (local address 127.30.71.2:39107)
0810 06:39:40.954836 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.954838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.954849 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:39:40.954991 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.954993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.955000 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.955071 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.955298 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.955865 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.956863 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.957222 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.957303 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.957314 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.957463 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.957501 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 06:39:40.957608 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.954694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:39107 (local address 127.30.71.1:52473)
0810 06:39:40.954764 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.954779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.954843 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.955075 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.955076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.955077 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.955156 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.955158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.955958 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.955959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.956760 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.956762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.957305 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.957306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.957378 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.957381 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.957454 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.957497 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 06:39:40.957609 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.954878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:60909 (local address 127.30.71.3:38297)
0810 06:39:40.954934 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.954935 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.955092 (+ 157us) server_negotiation.cc:407] Connection header received
0810 06:39:40.955169 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.955170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.955179 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.955224 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:40.955369 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.955927 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.956875 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.957238 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.957338 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.957349 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.957504 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.957540 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:39:40.957631 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.954792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:38297 (local address 127.30.71.1:60909)
0810 06:39:40.954967 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.955018 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:40.955100 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:40.955226 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:40.955226 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:40.955228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:40.955300 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.955302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.956017 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.956019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.956773 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:40.956784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.957346 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.957347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:40.957430 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.957433 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:40.957498 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 06:39:40.957536 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260810 06:39:40.958032 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10eb07922691415f91c828b768144fee" candidate_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c897fec3c449eeb351a303d67fe80b" is_pre_election: true
I20260810 06:39:40.958040 19011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10eb07922691415f91c828b768144fee" candidate_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e0dffcb6874a55982e49b136bcaf57" is_pre_election: true
I20260810 06:39:40.958163 19151 raft_consensus.cc:2454] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b62a7bc5d5e846409cad5b95bc60df9c in term 0.
I20260810 06:39:40.958194 19011 raft_consensus.cc:2454] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b62a7bc5d5e846409cad5b95bc60df9c in term 0.
I20260810 06:39:40.958413 18803 leader_election.cc:302] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c897fec3c449eeb351a303d67fe80b, b62a7bc5d5e846409cad5b95bc60df9c; no voters:
I20260810 06:39:40.958590 19221 raft_consensus.cc:2781] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:39:40.958665 19221 raft_consensus.cc:482] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:39:40.958684 19221 raft_consensus.cc:3037] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:40.959528 19221 raft_consensus.cc:504] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.959661 19221 leader_election.cc:288] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [CANDIDATE]: Term 1 election: Requested vote from peers 87e0dffcb6874a55982e49b136bcaf57 (127.30.71.2:39107), 03c897fec3c449eeb351a303d67fe80b (127.30.71.3:38297)
I20260810 06:39:40.959829 19011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10eb07922691415f91c828b768144fee" candidate_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e0dffcb6874a55982e49b136bcaf57"
I20260810 06:39:40.959873 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10eb07922691415f91c828b768144fee" candidate_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c897fec3c449eeb351a303d67fe80b"
I20260810 06:39:40.959933 19011 raft_consensus.cc:3037] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:40.959956 19151 raft_consensus.cc:3037] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:40.960850 19151 raft_consensus.cc:2454] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b62a7bc5d5e846409cad5b95bc60df9c in term 1.
I20260810 06:39:40.960850 19011 raft_consensus.cc:2454] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b62a7bc5d5e846409cad5b95bc60df9c in term 1.
I20260810 06:39:40.961037 18803 leader_election.cc:302] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c897fec3c449eeb351a303d67fe80b, b62a7bc5d5e846409cad5b95bc60df9c; no voters:
I20260810 06:39:40.961130 19221 raft_consensus.cc:2781] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:40.961273 19221 raft_consensus.cc:686] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [term 1 LEADER]: Becoming Leader. State: Replica: b62a7bc5d5e846409cad5b95bc60df9c, State: Running, Role: LEADER
I20260810 06:39:40.961386 19221 consensus_queue.cc:237] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } }
I20260810 06:39:40.962246 18717 catalog_manager.cc:5495] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c reported cstate change: term changed from 0 to 1, leader changed from <none> to b62a7bc5d5e846409cad5b95bc60df9c (127.30.71.1). New cstate: current_term: 1 leader_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b62a7bc5d5e846409cad5b95bc60df9c" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 46599 } health_report { overall_health: HEALTHY } } }
I20260810 06:39:40.967895 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.965706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.30.71.1:46599)
0810 06:39:40.965763 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:40.965765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:40.965772 (+ 7us) server_negotiation.cc:407] Connection header received
0810 06:39:40.965805 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:40.965806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:40.965822 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:40.965883 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:39:40.966001 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.966442 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.967285 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:40.967457 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:40.967509 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:40.967621 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:39:40.967700 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:39:40.967766 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:40.967818 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 06:39:40.967827 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 06:39:40.969573 19151 raft_consensus.cc:1261] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 1 FOLLOWER]: Refusing update from remote peer b62a7bc5d5e846409cad5b95bc60df9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:40.969573 19011 raft_consensus.cc:1261] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 1 FOLLOWER]: Refusing update from remote peer b62a7bc5d5e846409cad5b95bc60df9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:40.969950 19224 consensus_queue.cc:1035] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e0dffcb6874a55982e49b136bcaf57" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:40.970079 19221 consensus_queue.cc:1035] T 10eb07922691415f91c828b768144fee P b62a7bc5d5e846409cad5b95bc60df9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "03c897fec3c449eeb351a303d67fe80b" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 38297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:40.972463 19227 mvcc.cc:205] Tried to move back new op lower bound from 7316864946048802816 to 7316864946018508800. Current Snapshot: MvccSnapshot[applied={T|T < 7316864946048802816}]
I20260810 06:39:40.973124 19229 mvcc.cc:205] Tried to move back new op lower bound from 7316864946048802816 to 7316864946018508800. Current Snapshot: MvccSnapshot[applied={T|T < 7316864946048802816}]
I20260810 06:39:40.973363 19228 mvcc.cc:205] Tried to move back new op lower bound from 7316864946048802816 to 7316864946018508800. Current Snapshot: MvccSnapshot[applied={T|T < 7316864946048802816 or (T in {7316864946048802816})}]
I20260810 06:39:40.974625 18718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:40.974720 18718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:40.975459 18718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d55a5c76907434daa7a3e83b49a916d: Sending DeleteTablet for 3 replicas of tablet 10eb07922691415f91c828b768144fee
I20260810 06:39:40.975802 18850 tablet_service.cc:1514] Processing DeleteTablet for tablet 10eb07922691415f91c828b768144fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:40 UTC) from {username='slave'} at 127.0.0.1:39950
I20260810 06:39:40.976033 19128 tablet_service.cc:1514] Processing DeleteTablet for tablet 10eb07922691415f91c828b768144fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:40 UTC) from {username='slave'} at 127.0.0.1:34034
I20260810 06:39:40.976254 19239 tablet_replica.cc:330] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: stopping tablet replica
I20260810 06:39:40.976275 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 18786
I20260810 06:39:40.976413 18989 tablet_service.cc:1514] Processing DeleteTablet for tablet 10eb07922691415f91c828b768144fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:40 UTC) from {username='slave'} at 127.0.0.1:50884
I20260810 06:39:40.978493 19239 raft_consensus.cc:2229] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:40.978636 19239 raft_consensus.cc:2258] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:40.979641 19239 ts_tablet_manager.cc:1905] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:40.980566 19240 tablet_replica.cc:330] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: stopping tablet replica
I20260810 06:39:40.980675 19240 raft_consensus.cc:2229] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:40.980777 19240 raft_consensus.cc:2258] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:40.981235 19240 ts_tablet_manager.cc:1905] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:40.981246 19239 ts_tablet_manager.cc:1918] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:40.981338 19239 log.cc:1196] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/10eb07922691415f91c828b768144fee
I20260810 06:39:40.981590 19239 ts_tablet_manager.cc:1939] T 10eb07922691415f91c828b768144fee P 03c897fec3c449eeb351a303d67fe80b: Deleting consensus metadata
I20260810 06:39:40.981992 18702 catalog_manager.cc:4841] TS 03c897fec3c449eeb351a303d67fe80b (127.30.71.3:38297): tablet 10eb07922691415f91c828b768144fee (table test-table [id=6048cf48d07c48168675ffb4547100ca]) successfully deleted
I20260810 06:39:40.983235 19240 ts_tablet_manager.cc:1918] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:40.983316 19240 log.cc:1196] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/10eb07922691415f91c828b768144fee
I20260810 06:39:40.983573 19240 ts_tablet_manager.cc:1939] T 10eb07922691415f91c828b768144fee P 87e0dffcb6874a55982e49b136bcaf57: Deleting consensus metadata
I20260810 06:39:40.983999 18702 catalog_manager.cc:4841] TS 87e0dffcb6874a55982e49b136bcaf57 (127.30.71.2:39107): tablet 10eb07922691415f91c828b768144fee (table test-table [id=6048cf48d07c48168675ffb4547100ca]) successfully deleted
W20260810 06:39:40.985349 18701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:46599 (error 108)
I20260810 06:39:40.985605 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 18926
I20260810 06:39:40.985879 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:40.985740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:46599 (local address 127.0.0.1:39972)
0810 06:39:40.985804 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:40.985840 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:46599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:39:40.987212 18701 catalog_manager.cc:4584] TS b62a7bc5d5e846409cad5b95bc60df9c (127.30.71.1:46599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10eb07922691415f91c828b768144fee: Network error: Client connection negotiation failed: client connection to 127.30.71.1:46599: connect: Connection refused (error 111)
I20260810 06:39:40.992564 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 19064
I20260810 06:39:41.003556 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 18684
2026-08-10T06:39:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (697 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:41 dist-test-slave-w206 krb5kdc[19245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:41 dist-test-slave-w206 krb5kdc[19245](info): set up 2 sockets
Aug 10 06:39:41 dist-test-slave-w206 krb5kdc[19245](info): commencing operation
krb5kdc: starting...
W20260810 06:39:42.090176 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:39:42.226362 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:39181
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:39181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:42.326323 19258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:42.326531 19258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:42.326555 19258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:42.327653 19258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:42.327689 19258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:42.327703 19258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:42.327716 19258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:42.327728 19258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:42.329123 19258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:39181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:39181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:42.329319 19258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:42.329488 19258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:42.334378 19264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.334481 19266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.334380 19263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:42.334702 19258 server_base.cc:1034] running on GCE node
I20260810 06:39:42.335033 19258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:42.335345 19258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:42.336493 19258 hybrid_clock.cc:648] HybridClock initialized: now 1786343982336472 us; error 39 us; skew 500 ppm
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:42.339207 19258 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:39:42.339682 19258 webserver.cc:460] Webserver started at http://127.30.71.62:41541/ using document root <none> and password file <none>
I20260810 06:39:42.339951 19258 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:42.340020 19258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:42.340170 19258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:42.341248 19258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "605c7a5e52b7481292aa0fc9e0f272fa"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "e163b54d25bdabd09330782526562d9f"
server_key_iv: "fe1c622578f9bd070e38bf7bc55d4a7d"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.341639 19258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "605c7a5e52b7481292aa0fc9e0f272fa"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "e163b54d25bdabd09330782526562d9f"
server_key_iv: "fe1c622578f9bd070e38bf7bc55d4a7d"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.343044 19258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:42.343822 19274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.343992 19258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 06:39:42.344084 19258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "605c7a5e52b7481292aa0fc9e0f272fa"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "e163b54d25bdabd09330782526562d9f"
server_key_iv: "fe1c622578f9bd070e38bf7bc55d4a7d"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.344166 19258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:42.380582 19258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:42.382920 19258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:42.383118 19258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:42.386929 19258 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:39181
I20260810 06:39:42.387023 19333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:39181 every 8 connection(s)
I20260810 06:39:42.387393 19258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:42.388290 19334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:42.390615 19334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa: Bootstrap starting.
I20260810 06:39:42.391300 19334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:42.391587 19334 log.cc:824] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:42.392282 19334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa: No bootstrap required, opened a new log
I20260810 06:39:42.393025 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 19258
I20260810 06:39:42.393138 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:42.393349 31004 external_mini_cluster.cc:1428] Setting key cb499f670f9781fab91a520f0c7c07b5
I20260810 06:39:42.394413 19334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39181 } }
I20260810 06:39:42.394570 19334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:42.394615 19334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 605c7a5e52b7481292aa0fc9e0f272fa, State: Initialized, Role: FOLLOWER
I20260810 06:39:42.394767 19334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39181 } }
I20260810 06:39:42.394871 19334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:42.394917 19334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:42.394994 19334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:42.395865 19334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39181 } }
I20260810 06:39:42.395983 19334 leader_election.cc:302] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 605c7a5e52b7481292aa0fc9e0f272fa; no voters:
I20260810 06:39:42.396188 19334 leader_election.cc:288] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:42.396209 19339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:42.396327 19339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [term 1 LEADER]: Becoming Leader. State: Replica: 605c7a5e52b7481292aa0fc9e0f272fa, State: Running, Role: LEADER
I20260810 06:39:42.396410 19339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39181 } }
I20260810 06:39:42.396453 19334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:42.396749 19339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 605c7a5e52b7481292aa0fc9e0f272fa. Latest consensus state: current_term: 1 leader_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39181 } } }
I20260810 06:39:42.396817 19339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:42.397171 19341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605c7a5e52b7481292aa0fc9e0f272fa" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 39181 } } }
I20260810 06:39:42.397262 19341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:42.397545 19349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:42.398056 19349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:42.399647 19349 catalog_manager.cc:1347] Generated new cluster ID: b6f99ce11d7b4150b933091c0ebe160e
I20260810 06:39:42.399700 19349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:42.406103 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.393886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.30.71.62:39181)
0810 06:39:42.394107 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.394117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.394151 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:39:42.394375 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.394381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.394424 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.394562 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.395372 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.396447 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.397531 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.398137 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.398635 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.401128 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.401132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.401132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.401145 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.404528 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.404927 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.404930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.404931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.405044 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.405262 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.405263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.405263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.405503 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.405688 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.405755 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.405916 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
I20260810 06:39:42.411118 19349 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:42.411818 19349 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:42.421664 19349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa: Generated new TSK 0
I20260810 06:39:42.421862 19349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.71.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
Entry for principal kudu/127.30.71.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab.
I20260810 06:39:42.450495 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39181
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:42.534103 19361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:42.534361 19361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:42.534406 19361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:42.535650 19361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:42.535719 19361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:42.535765 19361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:39:42.537266 19361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.1.keytab
--principal=kudu/127.30.71.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:42.537559 19361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:42.537802 19361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:42.543135 19370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.543154 19368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.543177 19367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:42.543239 19361 server_base.cc:1034] running on GCE node
I20260810 06:39:42.543678 19361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:42.543956 19361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:42.545121 19361 hybrid_clock.cc:648] HybridClock initialized: now 1786343982545107 us; error 45 us; skew 500 ppm
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:42.548127 19361 init.cc:377] Logged in from keytab as kudu/127.30.71.1@KRBTEST.COM (short username kudu)
I20260810 06:39:42.548586 19361 webserver.cc:460] Webserver started at http://127.30.71.1:37989/ using document root <none> and password file <none>
I20260810 06:39:42.548849 19361 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:42.548921 19361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:42.549073 19361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:42.550243 19361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "985a7453ce22453bb128f9ab78e0e5ec"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "6a0694491576e910df27e6903acc6863"
server_key_iv: "d72238b1f1601d9a8a5e92390168241e"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.550631 19361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "985a7453ce22453bb128f9ab78e0e5ec"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "6a0694491576e910df27e6903acc6863"
server_key_iv: "d72238b1f1601d9a8a5e92390168241e"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.552095 19361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:42.552875 19379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.553066 19361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:42.553160 19361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "985a7453ce22453bb128f9ab78e0e5ec"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "6a0694491576e910df27e6903acc6863"
server_key_iv: "d72238b1f1601d9a8a5e92390168241e"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.553243 19361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:42.573568 19361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:42.576115 19361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:42.576298 19361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:42.576566 19361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:42.576898 19361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:42.576958 19361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.576996 19361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:42.577044 19361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.583007 19361 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:38643
I20260810 06:39:42.583048 19502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:38643 every 8 connection(s)
I20260810 06:39:42.583385 19361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:39:42.586643 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 19361
I20260810 06:39:42.586763 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:39:42.587047 31004 external_mini_cluster.cc:1428] Setting key 402cbe633f5cc33af50dccba10e64249
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.1@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:42.594822 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.584140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:54701 (local address 127.30.71.62:39181)
0810 06:39:42.584210 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.584212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.584423 (+ 211us) server_negotiation.cc:407] Connection header received
0810 06:39:42.584926 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.584927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.584941 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.585005 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.586030 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.586814 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.588287 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.588628 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.588718 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.591530 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.591531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.591532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.591542 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.593196 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.593750 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.593751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.593752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.593806 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.594140 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.594140 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.594141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.594245 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.594333 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.594683 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.594753 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:39:42.595171 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.584099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39181 (local address 127.30.71.1:54701)
0810 06:39:42.584302 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.584346 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.584842 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.585097 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.585101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.585696 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:42.585899 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.585903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.586924 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.586925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.588173 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.588176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.588728 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.588729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.588843 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.589426 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:42.589434 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:42.591411 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:42.593291 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.593302 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.593305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.593667 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.593885 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.593886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.593887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.594067 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.594351 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:42.594360 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:42.594539 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.594661 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.594865 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260810 06:39:42.595813 19503 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39181
I20260810 06:39:42.595930 19503 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:42.596148 19503 heartbeater.cc:511] Master 127.30.71.62:39181 requested a full tablet report, sending...
I20260810 06:39:42.596702 19295 ts_manager.cc:194] Registered new tserver with Master: 985a7453ce22453bb128f9ab78e0e5ec (127.30.71.1:38643)
WARNING: no policy specified for kudu/127.30.71.2@KRBTEST.COM; defaulting to no policy
I20260810 06:39:42.597414 19295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.1@KRBTEST.COM'} at 127.30.71.1:54701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
Entry for principal kudu/127.30.71.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab.
I20260810 06:39:42.615854 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39181
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:42.701378 19508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:42.701583 19508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:42.701607 19508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:42.702764 19508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:42.702809 19508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:42.702836 19508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:39:42.704383 19508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.2.keytab
--principal=kudu/127.30.71.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:42.704628 19508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:42.704867 19508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:42.710067 19517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.710062 19513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:42.710080 19508 server_base.cc:1034] running on GCE node
W20260810 06:39:42.710064 19515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:42.710461 19508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:42.710696 19508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:42.711848 19508 hybrid_clock.cc:648] HybridClock initialized: now 1786343982711831 us; error 34 us; skew 500 ppm
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:42.714322 19508 init.cc:377] Logged in from keytab as kudu/127.30.71.2@KRBTEST.COM (short username kudu)
I20260810 06:39:42.714821 19508 webserver.cc:460] Webserver started at http://127.30.71.2:45081/ using document root <none> and password file <none>
I20260810 06:39:42.715109 19508 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:42.715178 19508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:42.715317 19508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:42.716354 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "2b1a5d2f23e648829f21473fa98ae7df"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "1e3a87d5963ce1ae20dac041171c6238"
server_key_iv: "f5f8fdd7f21157cb891f4161b8d69ef4"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.716728 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "2b1a5d2f23e648829f21473fa98ae7df"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "1e3a87d5963ce1ae20dac041171c6238"
server_key_iv: "f5f8fdd7f21157cb891f4161b8d69ef4"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.718112 19508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:42.718819 19524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.719030 19508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:39:42.719137 19508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "2b1a5d2f23e648829f21473fa98ae7df"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "1e3a87d5963ce1ae20dac041171c6238"
server_key_iv: "f5f8fdd7f21157cb891f4161b8d69ef4"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.719226 19508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:42.730541 19508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:42.732561 19508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:42.732676 19508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:42.732853 19508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:42.733181 19508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:42.733242 19508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.733290 19508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:42.733319 19508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.739096 19508 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:42175
I20260810 06:39:42.739171 19645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:42175 every 8 connection(s)
I20260810 06:39:42.739534 19508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:39:42.741498 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 19508
I20260810 06:39:42.741585 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:39:42.741755 31004 external_mini_cluster.cc:1428] Setting key 3410adffbc16cb840af0ea6b3d364812
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.2@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:42.749856 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.740253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:50843 (local address 127.30.71.62:39181)
0810 06:39:42.740351 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.740353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.740494 (+ 141us) server_negotiation.cc:407] Connection header received
0810 06:39:42.740989 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.740990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.741000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.741053 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.742118 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.742833 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.743861 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.744176 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.744254 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.746697 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.746699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.746699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.746708 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.748460 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.748877 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.748878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.748879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.748947 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.749182 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.749183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.749184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.749336 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.749409 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.749649 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.749743 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:39:42.749886 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.740187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39181 (local address 127.30.71.2:50843)
0810 06:39:42.740336 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.740386 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.740905 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.741106 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.741109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.741773 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:42.742032 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.742037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.742924 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.742925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.743770 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.743772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.744275 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.744276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.744435 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.744967 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:42.744978 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:42.746609 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:42.748542 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.748548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.748550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.748770 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.749014 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.749015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.749015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.749114 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.749403 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:42.749408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:42.749517 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.749599 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.749716 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":99,"threads_started":1}
I20260810 06:39:42.750401 19646 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39181
I20260810 06:39:42.750506 19646 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:42.750711 19646 heartbeater.cc:511] Master 127.30.71.62:39181 requested a full tablet report, sending...
I20260810 06:39:42.751117 19295 ts_manager.cc:194] Registered new tserver with Master: 2b1a5d2f23e648829f21473fa98ae7df (127.30.71.2:42175)
WARNING: no policy specified for kudu/127.30.71.3@KRBTEST.COM; defaulting to no policy
I20260810 06:39:42.751531 19295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.2@KRBTEST.COM'} at 127.30.71.2:50843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
Entry for principal kudu/127.30.71.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab.
I20260810 06:39:42.770818 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39181
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:42.854460 19651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:42.854658 19651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:42.854681 19651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:42.855949 19651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:42.855991 19651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:42.856019 19651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:39:42.857525 19651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.3.keytab
--principal=kudu/127.30.71.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:39181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:42.857784 19651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:42.858016 19651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:42.863159 19657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.863173 19656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:42.863159 19659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:42.863539 19651 server_base.cc:1034] running on GCE node
I20260810 06:39:42.863725 19651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:42.863953 19651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:42.865087 19651 hybrid_clock.cc:648] HybridClock initialized: now 1786343982865050 us; error 56 us; skew 500 ppm
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:42.867766 19651 init.cc:377] Logged in from keytab as kudu/127.30.71.3@KRBTEST.COM (short username kudu)
I20260810 06:39:42.868248 19651 webserver.cc:460] Webserver started at http://127.30.71.3:33001/ using document root <none> and password file <none>
I20260810 06:39:42.868507 19651 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:42.868553 19651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:42.868721 19651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:42.869777 19651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "f51911d547a6414bb6d08c318f493cfe"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "73ade125e9993cc47385c2dd5e95eb11"
server_key_iv: "0d7b5f87eb1bfcc19bb882c5aed84157"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.870137 19651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "f51911d547a6414bb6d08c318f493cfe"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "73ade125e9993cc47385c2dd5e95eb11"
server_key_iv: "0d7b5f87eb1bfcc19bb882c5aed84157"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.871572 19651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:42.872407 19669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.872645 19651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:42.872718 19651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "f51911d547a6414bb6d08c318f493cfe"
format_stamp: "Formatted at 2026-08-10 06:39:42 on dist-test-slave-w206"
server_key: "73ade125e9993cc47385c2dd5e95eb11"
server_key_iv: "0d7b5f87eb1bfcc19bb882c5aed84157"
server_key_version: "encryptionkey@0"
I20260810 06:39:42.872810 19651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:42.903201 19651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:42.905309 19651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:42.905471 19651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:42.905719 19651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:42.906095 19651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:42.906150 19651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.906186 19651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:42.906208 19651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:42.914685 19651 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:37503
I20260810 06:39:42.914764 19788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:37503 every 8 connection(s)
I20260810 06:39:42.915160 19651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:39:42.916600 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 19651
I20260810 06:39:42.916723 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:39:42.916924 31004 external_mini_cluster.cc:1428] Setting key 5987cb0fc3b316ee59afe8f774bfc13b
Aug 10 06:39:42 dist-test-slave-w206 krb5kdc[19245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.3@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:39:42.925858 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.916056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:52513 (local address 127.30.71.62:39181)
0810 06:39:42.916141 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.916143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.916453 (+ 310us) server_negotiation.cc:407] Connection header received
0810 06:39:42.916988 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.916989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.917005 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.917092 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.918335 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.918850 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.919966 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.920149 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.920206 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.922779 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.922781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.922781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.922790 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.924390 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.924841 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.924843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.924843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.924909 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.925170 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.925171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.925172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.925303 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.925377 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.925668 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.925751 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:39:42.926064 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.915991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:39181 (local address 127.30.71.3:52513)
0810 06:39:42.916228 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.916268 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.916868 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.917185 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.917190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.918019 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:42.918234 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.918239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.918940 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.918941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.919879 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.919881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.920251 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.920252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.920382 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.920896 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:39:42.920906 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:42.922679 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:42.924496 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.924503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.924505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.924741 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.924984 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:39:42.924985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:39:42.924985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:39:42.925102 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:39:42.925379 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:42.925386 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:42.925533 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.925644 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.925825 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260810 06:39:42.926602 19789 heartbeater.cc:347] Connected to a master server at 127.30.71.62:39181
I20260810 06:39:42.926710 19789 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:42.926926 19789 heartbeater.cc:511] Master 127.30.71.62:39181 requested a full tablet report, sending...
I20260810 06:39:42.927369 19295 ts_manager.cc:194] Registered new tserver with Master: f51911d547a6414bb6d08c318f493cfe (127.30.71.3:37503)
I20260810 06:39:42.927754 19295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.71.3@KRBTEST.COM'} at 127.30.71.3:52513
I20260810 06:39:42.928810 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:39:42.936297 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.930156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.30.71.62:39181)
0810 06:39:42.930273 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.930276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.930309 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:39:42.930386 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.930387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.930396 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.930445 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:42.931059 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.931544 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.932275 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.932478 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.932552 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.933733 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:42.933734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:42.933735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:42.933741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:42.935060 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.935435 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.935437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.935437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.935500 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:42.935705 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:42.935706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:42.935706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:42.935859 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:42.935962 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.936108 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.936186 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 06:39:42.938040 19295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:39:42.939744 19295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:39:42.946805 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.942506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.30.71.2:42175)
0810 06:39:42.942832 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.942836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.942849 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:39:42.942890 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.942893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.942908 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.943074 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.943294 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.944014 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.945619 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.946369 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.946449 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.946516 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.946671 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.946718 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":85,"threads_started":1}
I20260810 06:39:42.946790 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.942328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38643 (local address 127.0.0.1:37418)
0810 06:39:42.942611 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.942620 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.942663 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.942959 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.942962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.942965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.943067 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.943073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.944478 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.944479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.945528 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.945530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.946478 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.946479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.946558 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.946563 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.946612 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.946672 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":84,"threads_started":1}
I20260810 06:39:42.947151 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.942441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37418 (local address 127.30.71.1:38643)
0810 06:39:42.942770 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.942773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.942782 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:39:42.942815 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.942817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.942828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.943157 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.943233 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.944375 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.945628 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.946385 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.946747 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.947002 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.947031 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.947074 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":86,"threads_started":1}
I20260810 06:39:42.947295 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.942120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:42175 (local address 127.0.0.1:53402)
0810 06:39:42.942399 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.942443 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.942509 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.943155 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.943156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.943158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.943207 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.943208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.944208 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.944209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.945504 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.945508 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.946447 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.946448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.946539 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.946559 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.946640 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.946696 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":53,"threads_started":1}
I20260810 06:39:42.947985 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.942143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:37503 (local address 127.0.0.1:46428)
0810 06:39:42.942501 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:42.942512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:42.942607 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:42.943046 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:42.943047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:42.943050 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:42.943117 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.943120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.945651 (+ 2531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.945651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.946502 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:42.946504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.947727 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.947729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:42.947796 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.947801 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:42.947873 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:39:42.947911 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":106,"threads_started":1}
I20260810 06:39:42.948022 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:42.942246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.30.71.3:37503)
0810 06:39:42.942443 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:42.942447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:42.942575 (+ 128us) server_negotiation.cc:407] Connection header received
0810 06:39:42.942764 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:42.942769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:42.942789 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:42.942914 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:42.943260 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.944360 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.946583 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:42.947639 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:42.947709 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:42.947759 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:42.947891 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:39:42.947933 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260810 06:39:42.947976 19433 tablet_service.cc:1467] Processing CreateTablet for tablet 241c3828d5d247a9881145c1561cbaa4 (DEFAULT_TABLE table=test-table [id=addcdac224ef49e4865bd207d89a5ca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:42.948017 19575 tablet_service.cc:1467] Processing CreateTablet for tablet 241c3828d5d247a9881145c1561cbaa4 (DEFAULT_TABLE table=test-table [id=addcdac224ef49e4865bd207d89a5ca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:42.948341 19433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241c3828d5d247a9881145c1561cbaa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:42.948341 19575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241c3828d5d247a9881145c1561cbaa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:42.951186 19808 tablet_bootstrap.cc:492] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: Bootstrap starting.
I20260810 06:39:42.951131 19807 tablet_bootstrap.cc:492] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: Bootstrap starting.
I20260810 06:39:42.951928 19808 tablet_bootstrap.cc:654] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:42.951934 19807 tablet_bootstrap.cc:654] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:42.951890 19719 tablet_service.cc:1467] Processing CreateTablet for tablet 241c3828d5d247a9881145c1561cbaa4 (DEFAULT_TABLE table=test-table [id=addcdac224ef49e4865bd207d89a5ca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:42.952301 19719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241c3828d5d247a9881145c1561cbaa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:42.952497 19807 log.cc:824] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:42.952564 19808 log.cc:824] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:42.953254 19808 tablet_bootstrap.cc:492] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: No bootstrap required, opened a new log
I20260810 06:39:42.953330 19807 tablet_bootstrap.cc:492] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: No bootstrap required, opened a new log
I20260810 06:39:42.953351 19808 ts_tablet_manager.cc:1397] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:42.953403 19807 ts_tablet_manager.cc:1397] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:42.954892 19808 raft_consensus.cc:348] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:42.955102 19808 raft_consensus.cc:374] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:42.955152 19808 raft_consensus.cc:729] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 985a7453ce22453bb128f9ab78e0e5ec, State: Initialized, Role: FOLLOWER
I20260810 06:39:42.955307 19808 consensus_queue.cc:260] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:42.955675 19808 ts_tablet_manager.cc:1428] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:39:42.955616 19807 raft_consensus.cc:348] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:42.955821 19807 raft_consensus.cc:374] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:42.955881 19807 raft_consensus.cc:729] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b1a5d2f23e648829f21473fa98ae7df, State: Initialized, Role: FOLLOWER
I20260810 06:39:42.956007 19807 consensus_queue.cc:260] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:42.956157 19812 tablet_bootstrap.cc:492] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Bootstrap starting.
I20260810 06:39:42.956313 19807 ts_tablet_manager.cc:1428] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:42.956355 19646 heartbeater.cc:503] Master 127.30.71.62:39181 was elected leader, sending a full tablet report...
I20260810 06:39:42.956445 19503 heartbeater.cc:503] Master 127.30.71.62:39181 was elected leader, sending a full tablet report...
I20260810 06:39:42.956879 19812 tablet_bootstrap.cc:654] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:42.957475 19812 log.cc:824] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:42.958328 19812 tablet_bootstrap.cc:492] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: No bootstrap required, opened a new log
I20260810 06:39:42.958387 19812 ts_tablet_manager.cc:1397] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:42.959764 19812 raft_consensus.cc:348] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:42.959914 19812 raft_consensus.cc:374] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:42.959939 19812 raft_consensus.cc:729] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51911d547a6414bb6d08c318f493cfe, State: Initialized, Role: FOLLOWER
I20260810 06:39:42.960050 19812 consensus_queue.cc:260] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:42.960271 19812 ts_tablet_manager.cc:1428] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:39:42.960340 19789 heartbeater.cc:503] Master 127.30.71.62:39181 was elected leader, sending a full tablet report...
W20260810 06:39:42.990123 19647 tablet.cc:2369] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:43.077572 19813 raft_consensus.cc:482] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:43.077703 19813 raft_consensus.cc:504] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:43.078207 19813 leader_election.cc:288] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 985a7453ce22453bb128f9ab78e0e5ec (127.30.71.1:38643), f51911d547a6414bb6d08c318f493cfe (127.30.71.3:37503)
I20260810 06:39:43.081250 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.078496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:53287 (local address 127.30.71.1:38643)
0810 06:39:43.078543 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.078545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.078573 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:39:43.078648 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.078649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.078662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.078724 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:43.078946 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.079501 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.080478 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.080884 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.080937 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.080946 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.081138 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.081182 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:39:43.081243 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.078434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38643 (local address 127.30.71.2:53287)
0810 06:39:43.078511 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.078524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.078570 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.078743 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.078744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.078746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:43.078861 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.078863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.079586 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.079588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.080374 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.080377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.080973 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.080974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.081056 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.081061 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.081109 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.081142 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 06:39:43.081319 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.078476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:35155 (local address 127.30.71.3:37503)
0810 06:39:43.078536 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.078539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.078784 (+ 245us) server_negotiation.cc:407] Connection header received
0810 06:39:43.078840 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.078841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.078851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.078912 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:43.079088 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.079591 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.080556 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.080942 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.081014 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.081024 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.081186 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.081231 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 06:39:43.081336 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.078436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:37503 (local address 127.30.71.2:35155)
0810 06:39:43.078706 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.078716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.078775 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.078924 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.078925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.078927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:43.079029 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.079031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.079684 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.079684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.080465 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.080468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.081033 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.081034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.081088 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.081110 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.081169 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.081205 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":78,"thread_start_us":111,"threads_started":1}
I20260810 06:39:43.081708 19453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241c3828d5d247a9881145c1561cbaa4" candidate_uuid: "2b1a5d2f23e648829f21473fa98ae7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "985a7453ce22453bb128f9ab78e0e5ec" is_pre_election: true
I20260810 06:39:43.081779 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241c3828d5d247a9881145c1561cbaa4" candidate_uuid: "2b1a5d2f23e648829f21473fa98ae7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51911d547a6414bb6d08c318f493cfe" is_pre_election: true
I20260810 06:39:43.081892 19453 raft_consensus.cc:2454] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b1a5d2f23e648829f21473fa98ae7df in term 0.
I20260810 06:39:43.081928 19739 raft_consensus.cc:2454] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b1a5d2f23e648829f21473fa98ae7df in term 0.
I20260810 06:39:43.082123 19526 leader_election.cc:302] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b1a5d2f23e648829f21473fa98ae7df, f51911d547a6414bb6d08c318f493cfe; no voters:
I20260810 06:39:43.082264 19813 raft_consensus.cc:2781] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:39:43.082345 19813 raft_consensus.cc:482] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:39:43.082379 19813 raft_consensus.cc:3037] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:43.083179 19813 raft_consensus.cc:504] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:43.083328 19813 leader_election.cc:288] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [CANDIDATE]: Term 1 election: Requested vote from peers 985a7453ce22453bb128f9ab78e0e5ec (127.30.71.1:38643), f51911d547a6414bb6d08c318f493cfe (127.30.71.3:37503)
I20260810 06:39:43.083549 19453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241c3828d5d247a9881145c1561cbaa4" candidate_uuid: "2b1a5d2f23e648829f21473fa98ae7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "985a7453ce22453bb128f9ab78e0e5ec"
I20260810 06:39:43.083648 19453 raft_consensus.cc:3037] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:43.083683 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241c3828d5d247a9881145c1561cbaa4" candidate_uuid: "2b1a5d2f23e648829f21473fa98ae7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51911d547a6414bb6d08c318f493cfe"
I20260810 06:39:43.083789 19739 raft_consensus.cc:3037] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 0 FOLLOWER]: Advancing to term 1
W20260810 06:39:43.084066 19504 tablet.cc:2369] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:43.084625 19453 raft_consensus.cc:2454] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b1a5d2f23e648829f21473fa98ae7df in term 1.
I20260810 06:39:43.084734 19739 raft_consensus.cc:2454] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b1a5d2f23e648829f21473fa98ae7df in term 1.
I20260810 06:39:43.084818 19529 leader_election.cc:302] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b1a5d2f23e648829f21473fa98ae7df, 985a7453ce22453bb128f9ab78e0e5ec; no voters:
I20260810 06:39:43.084911 19813 raft_consensus.cc:2781] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:43.085093 19813 raft_consensus.cc:686] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 1 LEADER]: Becoming Leader. State: Replica: 2b1a5d2f23e648829f21473fa98ae7df, State: Running, Role: LEADER
I20260810 06:39:43.085243 19813 consensus_queue.cc:237] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } }
I20260810 06:39:43.086179 19295 catalog_manager.cc:5495] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b1a5d2f23e648829f21473fa98ae7df (127.30.71.2). New cstate: current_term: 1 leader_uuid: "2b1a5d2f23e648829f21473fa98ae7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b1a5d2f23e648829f21473fa98ae7df" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 } health_report { overall_health: UNKNOWN } } }
I20260810 06:39:43.127794 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.125209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.30.71.2:42175)
0810 06:39:43.125336 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.125338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.125350 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:39:43.125387 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.125388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.125409 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.125493 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:39:43.125642 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.126120 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.126920 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.127178 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.127263 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.127457 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:39:43.127564 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:39:43.127656 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.127679 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.127701 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:39:43.129971 19453 raft_consensus.cc:1261] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 1 FOLLOWER]: Refusing update from remote peer 2b1a5d2f23e648829f21473fa98ae7df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:43.129971 19739 raft_consensus.cc:1261] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 1 FOLLOWER]: Refusing update from remote peer 2b1a5d2f23e648829f21473fa98ae7df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:43.130402 19817 consensus_queue.cc:1035] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [LEADER]: Connected to new peer: Peer: permanent_uuid: "985a7453ce22453bb128f9ab78e0e5ec" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 38643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:43.130558 19813 consensus_queue.cc:1035] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51911d547a6414bb6d08c318f493cfe" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 37503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:43.133391 19821 mvcc.cc:205] Tried to move back new op lower bound from 7316864954896990208 to 7316864954717691904. Current Snapshot: MvccSnapshot[applied={T|T < 7316864954896990208}]
I20260810 06:39:43.135337 19295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:43.135445 19295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:43.136219 19295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 605c7a5e52b7481292aa0fc9e0f272fa: Sending DeleteTablet for 3 replicas of tablet 241c3828d5d247a9881145c1561cbaa4
I20260810 06:39:43.136607 19719 tablet_service.cc:1514] Processing DeleteTablet for tablet 241c3828d5d247a9881145c1561cbaa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:43 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:46428
I20260810 06:39:43.136646 19575 tablet_service.cc:1514] Processing DeleteTablet for tablet 241c3828d5d247a9881145c1561cbaa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:43 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:53402
I20260810 06:39:43.136629 19433 tablet_service.cc:1514] Processing DeleteTablet for tablet 241c3828d5d247a9881145c1561cbaa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:43 UTC) from {username='kudu', principal='kudu/127.30.71.62@KRBTEST.COM'} at 127.0.0.1:37418
I20260810 06:39:43.136940 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 19361
I20260810 06:39:43.136981 19832 tablet_replica.cc:330] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec: stopping tablet replica
I20260810 06:39:43.137151 19832 raft_consensus.cc:2229] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:43.137337 19832 raft_consensus.cc:2258] T 241c3828d5d247a9881145c1561cbaa4 P 985a7453ce22453bb128f9ab78e0e5ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:43.138533 19831 tablet_replica.cc:330] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df: stopping tablet replica
I20260810 06:39:43.138736 19831 raft_consensus.cc:2229] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:39:43.139006 19831 raft_consensus.cc:2258] T 241c3828d5d247a9881145c1561cbaa4 P 2b1a5d2f23e648829f21473fa98ae7df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 06:39:43.145839 19278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:38643 (error 108)
I20260810 06:39:43.146075 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 19508
I20260810 06:39:43.146296 19830 tablet_replica.cc:330] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: stopping tablet replica
I20260810 06:39:43.146436 19830 raft_consensus.cc:2229] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:43.146550 19830 raft_consensus.cc:2258] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:43.147640 19830 ts_tablet_manager.cc:1905] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:43.148707 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.148557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:38643 (local address 127.0.0.1:37432)
0810 06:39:43.148616 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.148662 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:39:43.148839 19278 catalog_manager.cc:4584] TS 985a7453ce22453bb128f9ab78e0e5ec (127.30.71.1:38643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 241c3828d5d247a9881145c1561cbaa4: Network error: Client connection negotiation failed: client connection to 127.30.71.1:38643: connect: Connection refused (error 111)
I20260810 06:39:43.150098 19830 ts_tablet_manager.cc:1918] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:43.150200 19830 log.cc:1196] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/241c3828d5d247a9881145c1561cbaa4
I20260810 06:39:43.150516 19830 ts_tablet_manager.cc:1939] T 241c3828d5d247a9881145c1561cbaa4 P f51911d547a6414bb6d08c318f493cfe: Deleting consensus metadata
I20260810 06:39:43.151257 19276 catalog_manager.cc:4841] TS f51911d547a6414bb6d08c318f493cfe (127.30.71.3:37503): tablet 241c3828d5d247a9881145c1561cbaa4 (table test-table [id=addcdac224ef49e4865bd207d89a5ca0]) successfully deleted
I20260810 06:39:43.153573 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.153452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:42175 (local address 127.0.0.1:53420)
0810 06:39:43.153510 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.153543 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.2:42175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:39:43.153609 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 19651
I20260810 06:39:43.160584 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 19258
2026-08-10T06:39:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2159 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T06:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:43Z Disabled control of system clock
I20260810 06:39:43.192585 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41635
--webserver_interface=127.30.71.62
--webserver_port=0
--builtin_ntp_servers=127.30.71.20:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:41635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:43.275063 19837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:43.275298 19837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:43.275319 19837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:43.276417 19837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:43.276455 19837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:43.276467 19837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:43.276480 19837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:43.276492 19837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:43.277761 19837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:41635
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:43.277966 19837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:43.278204 19837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:43.283360 19843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:43.283344 19842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:43.283573 19837 server_base.cc:1034] running on GCE node
W20260810 06:39:43.283381 19846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:43.283932 19837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:43.284231 19837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:43.285395 19837 hybrid_clock.cc:648] HybridClock initialized: now 1786343983285384 us; error 29 us; skew 500 ppm
I20260810 06:39:43.286607 19837 webserver.cc:460] Webserver started at http://127.30.71.62:33265/ using document root <none> and password file <none>
I20260810 06:39:43.286859 19837 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:43.286932 19837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:43.287117 19837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:43.288234 19837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "956694bd360b49198dd189b27a90ea9d"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "bc9539a1406bc92c2eb7f62708cfb977"
server_key_iv: "d38543b21c8ff359f409db70d55deba1"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.288692 19837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "956694bd360b49198dd189b27a90ea9d"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "bc9539a1406bc92c2eb7f62708cfb977"
server_key_iv: "d38543b21c8ff359f409db70d55deba1"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.290030 19837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:43.290766 19853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.290951 19837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:39:43.291108 19837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "956694bd360b49198dd189b27a90ea9d"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "bc9539a1406bc92c2eb7f62708cfb977"
server_key_iv: "d38543b21c8ff359f409db70d55deba1"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.291194 19837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:43.322033 19837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:43.322408 19837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:43.322575 19837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:43.326728 19837 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:41635
I20260810 06:39:43.326771 19905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:41635 every 8 connection(s)
I20260810 06:39:43.327147 19837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:43.327557 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 19837
I20260810 06:39:43.327675 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:43.327903 31004 external_mini_cluster.cc:1428] Setting key 96bf138b6a41e306049ddc0d22e5935d
I20260810 06:39:43.328431 19906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:43.331760 19906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d: Bootstrap starting.
I20260810 06:39:43.332512 19906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:43.332872 19906 log.cc:824] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:43.333719 19906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d: No bootstrap required, opened a new log
I20260810 06:39:43.333730 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.328515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.30.71.62:41635)
0810 06:39:43.328747 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.328755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.328779 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:39:43.329229 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.329236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.329296 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.329414 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:43.329699 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.330579 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.331693 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.332155 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.332246 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.333045 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:43.333049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:43.333050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:43.333212 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:43.333246 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:43.333256 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:43.333311 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.333431 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.333512 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260810 06:39:43.335573 19906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956694bd360b49198dd189b27a90ea9d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41635 } }
I20260810 06:39:43.335709 19906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:43.335731 19906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956694bd360b49198dd189b27a90ea9d, State: Initialized, Role: FOLLOWER
I20260810 06:39:43.335855 19906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956694bd360b49198dd189b27a90ea9d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41635 } }
I20260810 06:39:43.335935 19906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:43.335990 19906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:43.336038 19906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:43.336658 19906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956694bd360b49198dd189b27a90ea9d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41635 } }
I20260810 06:39:43.336796 19906 leader_election.cc:302] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 956694bd360b49198dd189b27a90ea9d; no voters:
I20260810 06:39:43.337018 19906 leader_election.cc:288] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:43.337209 19912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:43.337332 19906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:43.337343 19912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [term 1 LEADER]: Becoming Leader. State: Replica: 956694bd360b49198dd189b27a90ea9d, State: Running, Role: LEADER
I20260810 06:39:43.337445 19912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956694bd360b49198dd189b27a90ea9d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41635 } }
I20260810 06:39:43.337901 19911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 956694bd360b49198dd189b27a90ea9d. Latest consensus state: current_term: 1 leader_uuid: "956694bd360b49198dd189b27a90ea9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956694bd360b49198dd189b27a90ea9d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41635 } } }
I20260810 06:39:43.338004 19911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:43.338167 19913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "956694bd360b49198dd189b27a90ea9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "956694bd360b49198dd189b27a90ea9d" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41635 } } }
I20260810 06:39:43.338253 19913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:43.338604 19920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:43.339148 19920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:43.343447 19920 catalog_manager.cc:1347] Generated new cluster ID: 1459c5b5ac54479b8c149506669f6d8d
I20260810 06:39:43.343541 19920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:43.363184 19920 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:43.363983 19920 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:43.375495 19920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d: Generated new TSK 0
I20260810 06:39:43.375739 19920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:43.378356 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.1:0
--local_ip_for_outbound_sockets=127.30.71.1
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41635
--builtin_ntp_servers=127.30.71.20:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:43.482640 19931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:43.482861 19931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:43.482885 19931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:43.484157 19931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:43.484210 19931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:43.484256 19931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.1
I20260810 06:39:43.485642 19931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.71.1
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.1
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:43.485936 19931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:43.486268 19931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:43.491595 19936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:43.491613 19931 server_base.cc:1034] running on GCE node
W20260810 06:39:43.491634 19939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:43.491847 19937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:43.492095 19931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:43.492427 19931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:43.493597 19931 hybrid_clock.cc:648] HybridClock initialized: now 1786343983493582 us; error 34 us; skew 500 ppm
I20260810 06:39:43.495064 19931 webserver.cc:460] Webserver started at http://127.30.71.1:41065/ using document root <none> and password file <none>
I20260810 06:39:43.495374 19931 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:43.495450 19931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:43.495597 19931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:43.496696 19931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data/instance:
uuid: "6988b7cee4c948c094e6d1c1cf322ccb"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "290f85be47e40fb44a25d7de164c4e4c"
server_key_iv: "1df1e6cc217db487413808ed181bd116"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.497082 19931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance:
uuid: "6988b7cee4c948c094e6d1c1cf322ccb"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "290f85be47e40fb44a25d7de164c4e4c"
server_key_iv: "1df1e6cc217db487413808ed181bd116"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.498454 19931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:43.499254 19947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.499414 19931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:43.499521 19931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/wal
uuid: "6988b7cee4c948c094e6d1c1cf322ccb"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "290f85be47e40fb44a25d7de164c4e4c"
server_key_iv: "1df1e6cc217db487413808ed181bd116"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.499598 19931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:43.509949 19931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:43.510286 19931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:43.510447 19931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:43.510690 19931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:43.511088 19931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:43.511143 19931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.511186 19931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:43.511227 19931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.517261 19931 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.1:42067
I20260810 06:39:43.517357 20068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.1:42067 every 8 connection(s)
I20260810 06:39:43.517700 19931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/data/info.pb
I20260810 06:39:43.523313 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.518496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.1:57641 (local address 127.30.71.62:41635)
0810 06:39:43.518563 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.518566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.518815 (+ 249us) server_negotiation.cc:407] Connection header received
0810 06:39:43.519475 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.519477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.519496 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.519564 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:43.520281 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.520809 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.521795 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.522012 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.522092 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.523009 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:43.523011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:43.523011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:43.523017 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:43.523034 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:43.523038 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:43.523087 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.523182 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.523229 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:39:43.523556 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.518486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41635 (local address 127.30.71.1:57641)
0810 06:39:43.518678 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.518714 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.519356 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.519639 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.519642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.519650 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:43.520184 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.520189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.520914 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.520915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.521696 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.521698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.522101 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.522102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.522238 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.522847 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:43.522848 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:43.522878 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:43.522902 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:43.523086 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:43.523092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:43.523095 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.523186 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.523303 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260810 06:39:43.524132 20069 heartbeater.cc:347] Connected to a master server at 127.30.71.62:41635
I20260810 06:39:43.524220 20069 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:43.524370 20069 heartbeater.cc:511] Master 127.30.71.62:41635 requested a full tablet report, sending...
I20260810 06:39:43.524868 19870 ts_manager.cc:194] Registered new tserver with Master: 6988b7cee4c948c094e6d1c1cf322ccb (127.30.71.1:42067)
I20260810 06:39:43.524896 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 19931
I20260810 06:39:43.525002 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-0/wal/instance
I20260810 06:39:43.525218 31004 external_mini_cluster.cc:1428] Setting key 0325af946dce259e600ffdf43c666466
I20260810 06:39:43.525774 19870 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.1:57641
I20260810 06:39:43.526291 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.2:0
--local_ip_for_outbound_sockets=127.30.71.2
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41635
--builtin_ntp_servers=127.30.71.20:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:39:43.609694 20072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:43.609920 20072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:43.609943 20072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:43.611156 20072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:43.611212 20072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:43.611238 20072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.2
I20260810 06:39:43.612587 20072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.71.2
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.2
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:43.612809 20072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:43.613080 20072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:43.618356 20080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:43.618378 20077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:43.618387 20078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:43.618752 20072 server_base.cc:1034] running on GCE node
I20260810 06:39:43.619027 20072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:43.619316 20072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:43.620497 20072 hybrid_clock.cc:648] HybridClock initialized: now 1786343983620472 us; error 43 us; skew 500 ppm
I20260810 06:39:43.621850 20072 webserver.cc:460] Webserver started at http://127.30.71.2:35375/ using document root <none> and password file <none>
I20260810 06:39:43.622130 20072 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:43.622201 20072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:43.622380 20072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:43.623605 20072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data/instance:
uuid: "4107373c4401449caddc4b4e78139389"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "946c721911a3b13a562d1aa00ff1bbc7"
server_key_iv: "ef193bb5c75db392652c782f193e7eeb"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.623989 20072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance:
uuid: "4107373c4401449caddc4b4e78139389"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "946c721911a3b13a562d1aa00ff1bbc7"
server_key_iv: "ef193bb5c75db392652c782f193e7eeb"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.625396 20072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:39:43.626206 20088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.626508 20072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:43.626614 20072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal
uuid: "4107373c4401449caddc4b4e78139389"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "946c721911a3b13a562d1aa00ff1bbc7"
server_key_iv: "ef193bb5c75db392652c782f193e7eeb"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.626691 20072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:43.652385 20072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:43.652740 20072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:43.652922 20072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:43.653178 20072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:43.653538 20072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:43.653595 20072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.653640 20072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:43.653666 20072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.659595 20072 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.2:41859
I20260810 06:39:43.659637 20211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.2:41859 every 8 connection(s)
I20260810 06:39:43.660003 20072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/data/info.pb
I20260810 06:39:43.662017 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 20072
I20260810 06:39:43.662165 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal/instance
I20260810 06:39:43.662424 31004 external_mini_cluster.cc:1428] Setting key be4658333b899b107c07308a25db91ed
I20260810 06:39:43.663533 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.71.3:0
--local_ip_for_outbound_sockets=127.30.71.3
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41635
--builtin_ntp_servers=127.30.71.20:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:39:43.666018 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.660758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.2:50257 (local address 127.30.71.62:41635)
0810 06:39:43.660833 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.660835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.661098 (+ 263us) server_negotiation.cc:407] Connection header received
0810 06:39:43.661673 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.661675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.661688 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.661754 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:43.662627 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.663313 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.664462 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.664672 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.664762 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.665733 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:43.665735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:43.665735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:43.665740 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:43.665756 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:43.665760 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:43.665815 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.665930 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.665956 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:39:43.666289 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.660741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41635 (local address 127.30.71.2:50257)
0810 06:39:43.660967 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.661004 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.661564 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.661836 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.661840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.661849 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:43.662483 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.662491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.663421 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.663422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.664365 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.664368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.664777 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.664778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.664958 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.665543 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:43.665543 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:43.665586 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:43.665618 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:43.665838 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:43.665844 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:43.665847 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.665922 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.666053 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260810 06:39:43.666894 20212 heartbeater.cc:347] Connected to a master server at 127.30.71.62:41635
I20260810 06:39:43.667026 20212 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:43.667240 20212 heartbeater.cc:511] Master 127.30.71.62:41635 requested a full tablet report, sending...
I20260810 06:39:43.667663 19870 ts_manager.cc:194] Registered new tserver with Master: 4107373c4401449caddc4b4e78139389 (127.30.71.2:41859)
I20260810 06:39:43.668040 19870 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.2:50257
W20260810 06:39:43.750000 20215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:43.750216 20215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:43.750237 20215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:43.751434 20215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:43.751477 20215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:43.751502 20215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.71.3
I20260810 06:39:43.752853 20215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.71.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.71.3
--webserver_port=0
--tserver_master_addrs=127.30.71.62:41635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.71.3
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:43.753118 20215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:43.753428 20215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:39:43.758687 20215 server_base.cc:1034] running on GCE node
W20260810 06:39:43.758632 20223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:43.758657 20220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:43.758657 20221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:43.759095 20215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:43.759326 20215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:43.760514 20215 hybrid_clock.cc:648] HybridClock initialized: now 1786343983760500 us; error 31 us; skew 500 ppm
I20260810 06:39:43.761804 20215 webserver.cc:460] Webserver started at http://127.30.71.3:39857/ using document root <none> and password file <none>
I20260810 06:39:43.762050 20215 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:43.762121 20215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:43.762297 20215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:43.763509 20215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data/instance:
uuid: "32a5df533a0b48e9937c8774770d3892"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "f4d8fa05660fa4af24f52992ae0f0f2a"
server_key_iv: "07874fed24b6ad4576952a9d0b3faee9"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.763870 20215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance:
uuid: "32a5df533a0b48e9937c8774770d3892"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "f4d8fa05660fa4af24f52992ae0f0f2a"
server_key_iv: "07874fed24b6ad4576952a9d0b3faee9"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.765210 20215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:43.765993 20233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.766165 20215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 06:39:43.766270 20215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal
uuid: "32a5df533a0b48e9937c8774770d3892"
format_stamp: "Formatted at 2026-08-10 06:39:43 on dist-test-slave-w206"
server_key: "f4d8fa05660fa4af24f52992ae0f0f2a"
server_key_iv: "07874fed24b6ad4576952a9d0b3faee9"
server_key_version: "encryptionkey@0"
I20260810 06:39:43.766364 20215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:43.782752 20215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:43.783121 20215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:43.783290 20215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:43.783546 20215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:39:43.783907 20215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:39:43.783942 20215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.783967 20215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:39:43.783978 20215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:43.789934 20215 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.3:45213
I20260810 06:39:43.789969 20356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.3:45213 every 8 connection(s)
I20260810 06:39:43.790349 20215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/data/info.pb
I20260810 06:39:43.795707 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.791205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:37215 (local address 127.30.71.62:41635)
0810 06:39:43.791266 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.791268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.791487 (+ 219us) server_negotiation.cc:407] Connection header received
0810 06:39:43.791969 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.791970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.791980 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.792033 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:43.792703 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.793199 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.794195 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.794391 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.794531 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.795470 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:43.795471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:43.795472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:43.795475 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:43.795492 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:43.795496 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:43.795540 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.795622 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.795644 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:39:43.795957 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.791189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:41635 (local address 127.30.71.3:37215)
0810 06:39:43.791363 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.791396 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.791883 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.792088 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.792091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.792099 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:39:43.792627 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.792632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.793275 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.793275 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.794118 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.794120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.794558 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.794559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.794684 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.795343 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:39:43.795344 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:39:43.795372 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:39:43.795395 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:39:43.795553 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:39:43.795557 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:39:43.795560 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.795612 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.795693 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260810 06:39:43.796465 20357 heartbeater.cc:347] Connected to a master server at 127.30.71.62:41635
I20260810 06:39:43.796566 20357 heartbeater.cc:464] Registering TS with master...
I20260810 06:39:43.796746 20357 heartbeater.cc:511] Master 127.30.71.62:41635 requested a full tablet report, sending...
I20260810 06:39:43.797143 19870 ts_manager.cc:194] Registered new tserver with Master: 32a5df533a0b48e9937c8774770d3892 (127.30.71.3:45213)
I20260810 06:39:43.797506 19870 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.71.3:37215
I20260810 06:39:43.800171 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 20215
I20260810 06:39:43.800299 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal/instance
I20260810 06:39:43.800514 31004 external_mini_cluster.cc:1428] Setting key def2d02f4c258e850edf03b884252500
I20260810 06:39:43.801563 31004 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:39:43.805703 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.803071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56798 (local address 127.30.71.62:41635)
0810 06:39:43.803130 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.803132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.803212 (+ 80us) server_negotiation.cc:407] Connection header received
0810 06:39:43.803247 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.803248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.803256 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.803316 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:43.803448 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.803910 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.804647 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.804827 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.804880 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.805495 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:43.805496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:43.805496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:43.805500 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:43.805513 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:43.805517 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:43.805559 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.805626 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.805645 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:39:43.807507 19870 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:39:43.809103 19870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:39:43.815968 20367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.811313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41859 (local address 127.0.0.1:55530)
0810 06:39:43.811814 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.811824 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.811872 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.812077 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.812080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.812083 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:43.812157 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.812158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.813507 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.813508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.814582 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.814585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.815667 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.815668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.815749 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.815763 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.815833 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.815884 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":47,"thread_start_us":84,"threads_started":1}
I20260810 06:39:43.816146 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.811384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38864 (local address 127.30.71.1:42067)
0810 06:39:43.811544 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.811549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.811941 (+ 392us) server_negotiation.cc:407] Connection header received
0810 06:39:43.811986 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.811988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.812006 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.812072 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:43.812271 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.813409 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.814800 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.815497 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.815555 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.815628 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.816016 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.816062 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260810 06:39:43.816247 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.811537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55530 (local address 127.30.71.2:41859)
0810 06:39:43.811726 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.811730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.811868 (+ 138us) server_negotiation.cc:407] Connection header received
0810 06:39:43.811954 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.811957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.811974 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.812056 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:43.812225 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.813404 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.814679 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.815446 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.815530 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.815628 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.816099 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.816150 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260810 06:39:43.816313 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.811293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42067 (local address 127.0.0.1:38864)
0810 06:39:43.811563 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.811603 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.811705 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.812139 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.812140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.812143 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:43.812206 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.812208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.813486 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.813487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.814706 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.814708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.815777 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.815779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.815843 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.815847 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.815908 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.815948 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":37,"thread_start_us":65,"threads_started":1}
I20260810 06:39:43.816942 20139 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7ab68e1fea475495bd132906b44c63 (DEFAULT_TABLE table=test-table [id=17782ea8a316474db052cde5809a6458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:43.817364 20139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7ab68e1fea475495bd132906b44c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:43.818159 19997 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7ab68e1fea475495bd132906b44c63 (DEFAULT_TABLE table=test-table [id=17782ea8a316474db052cde5809a6458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:43.818495 19997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7ab68e1fea475495bd132906b44c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:43.818578 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.811535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.30.71.3:45213)
0810 06:39:43.811827 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.811831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.812357 (+ 526us) server_negotiation.cc:407] Connection header received
0810 06:39:43.812424 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.812426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.812441 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.812511 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:43.812605 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.813287 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.815665 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.817235 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.817311 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.817402 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.818421 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.818473 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":89,"threads_started":1}
I20260810 06:39:43.818722 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.811321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.3:45213 (local address 127.0.0.1:60706)
0810 06:39:43.812304 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.812313 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.812356 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.812517 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.812518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.812519 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:43.812554 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.812555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.813372 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.813373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.814711 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.814713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.817341 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.817342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.818167 (+ 825us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.818173 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.818242 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.818286 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"mutex_wait_us":71,"thread_start_us":84,"threads_started":1}
I20260810 06:39:43.819713 20286 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7ab68e1fea475495bd132906b44c63 (DEFAULT_TABLE table=test-table [id=17782ea8a316474db052cde5809a6458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:39:43.820041 20286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7ab68e1fea475495bd132906b44c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:43.820960 20371 tablet_bootstrap.cc:492] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Bootstrap starting.
I20260810 06:39:43.821713 20371 tablet_bootstrap.cc:654] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:43.822057 20372 tablet_bootstrap.cc:492] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb: Bootstrap starting.
I20260810 06:39:43.822069 20371 log.cc:824] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:43.822717 20372 tablet_bootstrap.cc:654] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:43.822952 20371 tablet_bootstrap.cc:492] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: No bootstrap required, opened a new log
I20260810 06:39:43.823053 20371 ts_tablet_manager.cc:1397] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:39:43.823081 20372 log.cc:824] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:43.823800 20372 tablet_bootstrap.cc:492] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb: No bootstrap required, opened a new log
I20260810 06:39:43.823884 20372 ts_tablet_manager.cc:1397] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 06:39:43.825138 20371 raft_consensus.cc:348] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.825273 20371 raft_consensus.cc:374] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:43.825297 20371 raft_consensus.cc:729] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4107373c4401449caddc4b4e78139389, State: Initialized, Role: FOLLOWER
I20260810 06:39:43.825418 20371 consensus_queue.cc:260] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.825681 20371 ts_tablet_manager.cc:1428] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:39:43.825740 20212 heartbeater.cc:503] Master 127.30.71.62:41635 was elected leader, sending a full tablet report...
I20260810 06:39:43.825901 20374 tablet_bootstrap.cc:492] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Bootstrap starting.
I20260810 06:39:43.826188 20372 raft_consensus.cc:348] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.826341 20372 raft_consensus.cc:374] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:43.826385 20372 raft_consensus.cc:729] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6988b7cee4c948c094e6d1c1cf322ccb, State: Initialized, Role: FOLLOWER
I20260810 06:39:43.826498 20374 tablet_bootstrap.cc:654] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:43.826493 20372 consensus_queue.cc:260] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.826761 20372 ts_tablet_manager.cc:1428] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 06:39:43.826828 20069 heartbeater.cc:503] Master 127.30.71.62:41635 was elected leader, sending a full tablet report...
I20260810 06:39:43.827152 20374 log.cc:824] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:43.827984 20374 tablet_bootstrap.cc:492] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: No bootstrap required, opened a new log
I20260810 06:39:43.828061 20374 ts_tablet_manager.cc:1397] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:39:43.830089 20374 raft_consensus.cc:348] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.830240 20374 raft_consensus.cc:374] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:43.830286 20374 raft_consensus.cc:729] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a5df533a0b48e9937c8774770d3892, State: Initialized, Role: FOLLOWER
I20260810 06:39:43.830400 20374 consensus_queue.cc:260] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.830648 20374 ts_tablet_manager.cc:1428] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:39:43.830735 20357 heartbeater.cc:503] Master 127.30.71.62:41635 was elected leader, sending a full tablet report...
W20260810 06:39:43.910557 20213 tablet.cc:2369] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:39:43.923038 20379 raft_consensus.cc:482] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:39:43.923206 20379 raft_consensus.cc:504] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.923607 20379 leader_election.cc:288] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6988b7cee4c948c094e6d1c1cf322ccb (127.30.71.1:42067), 4107373c4401449caddc4b4e78139389 (127.30.71.2:41859)
I20260810 06:39:43.926652 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.923843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42067 (local address 127.30.71.3:57581)
0810 06:39:43.923922 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.923935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.923987 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.924182 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.924183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.924185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:43.924265 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.924268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.925013 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.925014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.925793 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.925795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.926368 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.926370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.926448 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.926451 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.926494 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.926528 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 06:39:43.926659 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.923938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:57581 (local address 127.30.71.1:42067)
0810 06:39:43.923988 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.923991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.924045 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:39:43.924078 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.924079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.924093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.924153 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:43.924396 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.924916 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.925894 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.926277 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.926335 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.926344 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.926523 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.926576 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:39:43.926700 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.923945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.71.3:59781 (local address 127.30.71.2:41859)
0810 06:39:43.923988 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.923991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.924166 (+ 175us) server_negotiation.cc:407] Connection header received
0810 06:39:43.924199 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.924200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.924213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.924276 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:39:43.924487 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.924946 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.925950 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.926291 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.926383 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.926392 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.926580 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.926626 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:39:43.926764 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.923843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.2:41859 (local address 127.30.71.3:59781)
0810 06:39:43.924075 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.924086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:39:43.924149 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:39:43.924312 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:39:43.924313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:39:43.924314 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:39:43.924405 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.924407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.925064 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.925065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.925826 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:39:43.925827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.926406 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.926407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:39:43.926496 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.926501 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:39:43.926561 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 06:39:43.926595 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":49,"thread_start_us":105,"threads_started":1}
I20260810 06:39:43.927090 20019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7ab68e1fea475495bd132906b44c63" candidate_uuid: "32a5df533a0b48e9937c8774770d3892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" is_pre_election: true
I20260810 06:39:43.927088 20163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7ab68e1fea475495bd132906b44c63" candidate_uuid: "32a5df533a0b48e9937c8774770d3892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4107373c4401449caddc4b4e78139389" is_pre_election: true
I20260810 06:39:43.927268 20019 raft_consensus.cc:2454] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a5df533a0b48e9937c8774770d3892 in term 0.
I20260810 06:39:43.927281 20163 raft_consensus.cc:2454] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a5df533a0b48e9937c8774770d3892 in term 0.
I20260810 06:39:43.927527 20236 leader_election.cc:302] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a5df533a0b48e9937c8774770d3892, 4107373c4401449caddc4b4e78139389; no voters:
I20260810 06:39:43.927661 20379 raft_consensus.cc:2781] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:39:43.927716 20379 raft_consensus.cc:482] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:39:43.927750 20379 raft_consensus.cc:3037] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:43.928632 20379 raft_consensus.cc:504] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.928790 20379 leader_election.cc:288] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [CANDIDATE]: Term 1 election: Requested vote from peers 6988b7cee4c948c094e6d1c1cf322ccb (127.30.71.1:42067), 4107373c4401449caddc4b4e78139389 (127.30.71.2:41859)
I20260810 06:39:43.928933 20019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7ab68e1fea475495bd132906b44c63" candidate_uuid: "32a5df533a0b48e9937c8774770d3892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6988b7cee4c948c094e6d1c1cf322ccb"
I20260810 06:39:43.928954 20163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7ab68e1fea475495bd132906b44c63" candidate_uuid: "32a5df533a0b48e9937c8774770d3892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4107373c4401449caddc4b4e78139389"
I20260810 06:39:43.929030 20163 raft_consensus.cc:3037] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:43.929042 20019 raft_consensus.cc:3037] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:43.929876 20163 raft_consensus.cc:2454] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a5df533a0b48e9937c8774770d3892 in term 1.
I20260810 06:39:43.929883 20019 raft_consensus.cc:2454] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a5df533a0b48e9937c8774770d3892 in term 1.
I20260810 06:39:43.930071 20235 leader_election.cc:302] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a5df533a0b48e9937c8774770d3892, 6988b7cee4c948c094e6d1c1cf322ccb; no voters:
I20260810 06:39:43.930189 20379 raft_consensus.cc:2781] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:43.930377 20379 raft_consensus.cc:686] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 1 LEADER]: Becoming Leader. State: Replica: 32a5df533a0b48e9937c8774770d3892, State: Running, Role: LEADER
I20260810 06:39:43.930473 20379 consensus_queue.cc:237] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } }
I20260810 06:39:43.931313 19870 catalog_manager.cc:5495] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32a5df533a0b48e9937c8774770d3892 (127.30.71.3). New cstate: current_term: 1 leader_uuid: "32a5df533a0b48e9937c8774770d3892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a5df533a0b48e9937c8774770d3892" member_type: VOTER last_known_addr { host: "127.30.71.3" port: 45213 } health_report { overall_health: HEALTHY } } }
I20260810 06:39:43.961727 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.959128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.30.71.3:45213)
0810 06:39:43.959196 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:43.959198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:43.959208 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:39:43.959299 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:43.959299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:43.959321 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:43.959413 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:39:43.959560 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.960057 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.960861 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:43.961096 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:43.961176 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:43.961357 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:39:43.961471 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:39:43.961571 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:43.961590 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 06:39:43.961609 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:39:43.963742 20163 raft_consensus.cc:1261] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 1 FOLLOWER]: Refusing update from remote peer 32a5df533a0b48e9937c8774770d3892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:43.963749 20019 raft_consensus.cc:1261] T 8b7ab68e1fea475495bd132906b44c63 P 6988b7cee4c948c094e6d1c1cf322ccb [term 1 FOLLOWER]: Refusing update from remote peer 32a5df533a0b48e9937c8774770d3892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:39:43.964121 20379 consensus_queue.cc:1035] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4107373c4401449caddc4b4e78139389" member_type: VOTER last_known_addr { host: "127.30.71.2" port: 41859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:43.964274 20382 consensus_queue.cc:1035] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6988b7cee4c948c094e6d1c1cf322ccb" member_type: VOTER last_known_addr { host: "127.30.71.1" port: 42067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:39:43.966485 20385 mvcc.cc:205] Tried to move back new op lower bound from 7316864958312501248 to 7316864958179868672. Current Snapshot: MvccSnapshot[applied={T|T < 7316864958312501248}]
I20260810 06:39:43.967945 20386 mvcc.cc:205] Tried to move back new op lower bound from 7316864958312501248 to 7316864958179868672. Current Snapshot: MvccSnapshot[applied={T|T < 7316864958312501248 or (T in {7316864958312501248})}]
I20260810 06:39:43.968581 20387 mvcc.cc:205] Tried to move back new op lower bound from 7316864958312501248 to 7316864958179868672. Current Snapshot: MvccSnapshot[applied={T|T < 7316864958312501248 or (T in {7316864958312501248})}]
I20260810 06:39:43.968880 19868 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:43.968981 19868 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:39:43.969815 19868 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 956694bd360b49198dd189b27a90ea9d: Sending DeleteTablet for 3 replicas of tablet 8b7ab68e1fea475495bd132906b44c63
I20260810 06:39:43.970162 19997 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7ab68e1fea475495bd132906b44c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:43 UTC) from {username='slave'} at 127.0.0.1:38864
I20260810 06:39:43.970392 20286 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7ab68e1fea475495bd132906b44c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:43 UTC) from {username='slave'} at 127.0.0.1:60706
I20260810 06:39:43.970441 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 19931
I20260810 06:39:43.974071 20139 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7ab68e1fea475495bd132906b44c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:39:43 UTC) from {username='slave'} at 127.0.0.1:55530
I20260810 06:39:43.974243 20396 tablet_replica.cc:330] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: stopping tablet replica
I20260810 06:39:43.974320 20397 tablet_replica.cc:330] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: stopping tablet replica
I20260810 06:39:43.974350 20396 raft_consensus.cc:2229] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:39:43.974433 20397 raft_consensus.cc:2229] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:39:43.974448 20396 raft_consensus.cc:2258] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:43.974545 20397 raft_consensus.cc:2258] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:39:43.975075 20397 ts_tablet_manager.cc:1905] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:43.977056 20397 ts_tablet_manager.cc:1918] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:43.977138 20397 log.cc:1196] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-1/wal/wals/8b7ab68e1fea475495bd132906b44c63
I20260810 06:39:43.977399 20397 ts_tablet_manager.cc:1939] T 8b7ab68e1fea475495bd132906b44c63 P 4107373c4401449caddc4b4e78139389: Deleting consensus metadata
I20260810 06:39:43.977891 19856 catalog_manager.cc:4841] TS 4107373c4401449caddc4b4e78139389 (127.30.71.2:41859): tablet 8b7ab68e1fea475495bd132906b44c63 (table test-table [id=17782ea8a316474db052cde5809a6458]) successfully deleted
W20260810 06:39:43.978319 19855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.71.1:42067 (error 108)
I20260810 06:39:43.978780 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:43.978658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.1:42067 (local address 127.0.0.1:38868)
0810 06:39:43.978700 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 06:39:43.978740 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.1:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260810 06:39:43.978925 19855 catalog_manager.cc:4584] TS 6988b7cee4c948c094e6d1c1cf322ccb (127.30.71.1:42067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b7ab68e1fea475495bd132906b44c63: Network error: Client connection negotiation failed: client connection to 127.30.71.1:42067: connect: Connection refused (error 111)
I20260810 06:39:43.979100 20396 ts_tablet_manager.cc:1905] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:39:43.979337 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 20072
I20260810 06:39:43.988812 20396 ts_tablet_manager.cc:1918] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:39:43.988919 20396 log.cc:1196] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Deleting WAL directory at /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786343734291041-31004-0/minicluster-data/ts-2/wal/wals/8b7ab68e1fea475495bd132906b44c63
I20260810 06:39:43.989195 20396 ts_tablet_manager.cc:1939] T 8b7ab68e1fea475495bd132906b44c63 P 32a5df533a0b48e9937c8774770d3892: Deleting consensus metadata
I20260810 06:39:43.989773 19854 catalog_manager.cc:4841] TS 32a5df533a0b48e9937c8774770d3892 (127.30.71.3:45213): tablet 8b7ab68e1fea475495bd132906b44c63 (table test-table [id=17782ea8a316474db052cde5809a6458]) successfully deleted
I20260810 06:39:43.992118 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 20215
I20260810 06:39:44.004175 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 19837
2026-08-10T06:39:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (841 ms)
[----------] 6 tests from EncryptionPolicyTest (8640 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:44 dist-test-slave-w206 krb5kdc[20403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:44 dist-test-slave-w206 krb5kdc[20403](info): set up 2 sockets
Aug 10 06:39:44 dist-test-slave-w206 krb5kdc[20403](info): commencing operation
krb5kdc: starting...
W20260810 06:39:45.086879 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:45 dist-test-slave-w206 krb5kdc[20403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:45.218063 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:45.301397 20417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:45.301661 20417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:45.301698 20417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:45.302887 20417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:45.302960 20417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:45.302984 20417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:45.303032 20417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:45.303061 20417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:45.304572 20417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:45.304853 20417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:45.305086 20417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:45.310086 20426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:45.310242 20417 server_base.cc:1034] running on GCE node
W20260810 06:39:45.310119 20423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:45.310086 20424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:45.310611 20417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:45.310920 20417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:45.312108 20417 hybrid_clock.cc:648] HybridClock initialized: now 1786343985312096 us; error 32 us; skew 500 ppm
Aug 10 06:39:45 dist-test-slave-w206 krb5kdc[20403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:45.314728 20417 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:45.315263 20417 webserver.cc:460] Webserver started at http://127.0.0.1:36645/ using document root <none> and password file <none>
I20260810 06:39:45.315462 20417 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:45.315505 20417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:45.315596 20417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:45.316608 20417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "2b7366b68bad4e05ad701f19e4f85538"
format_stamp: "Formatted at 2026-08-10 06:39:45 on dist-test-slave-w206"
server_key: "f981b975bf9f9f232b7754f8f020bc3c"
server_key_iv: "16827347cd8c622fff7c11d10b0a062f"
server_key_version: "encryptionkey@0"
I20260810 06:39:45.316912 20417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "2b7366b68bad4e05ad701f19e4f85538"
format_stamp: "Formatted at 2026-08-10 06:39:45 on dist-test-slave-w206"
server_key: "f981b975bf9f9f232b7754f8f020bc3c"
server_key_iv: "16827347cd8c622fff7c11d10b0a062f"
server_key_version: "encryptionkey@0"
I20260810 06:39:45.318177 20417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:45.319031 20434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:45.319234 20417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:45.319360 20417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "2b7366b68bad4e05ad701f19e4f85538"
format_stamp: "Formatted at 2026-08-10 06:39:45 on dist-test-slave-w206"
server_key: "f981b975bf9f9f232b7754f8f020bc3c"
server_key_iv: "16827347cd8c622fff7c11d10b0a062f"
server_key_version: "encryptionkey@0"
I20260810 06:39:45.319433 20417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:45.335673 20417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:45.338050 20417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:45.338215 20417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:45.342384 20417 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44701
I20260810 06:39:45.342464 20490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44701 every 8 connection(s)
I20260810 06:39:45.342828 20417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:45.343698 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 20417
I20260810 06:39:45.343839 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:45.343824 20491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:45.344045 31004 external_mini_cluster.cc:1428] Setting key d3ab935f95b5b509015d7ed2da0a9616
I20260810 06:39:45.347393 20491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538: Bootstrap starting.
I20260810 06:39:45.348099 20491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:45.348400 20491 log.cc:824] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:45.349188 20491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538: No bootstrap required, opened a new log
Aug 10 06:39:45 dist-test-slave-w206 krb5kdc[20403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:39:45.351122 20491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7366b68bad4e05ad701f19e4f85538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 06:39:45.351277 20491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:45.351329 20491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7366b68bad4e05ad701f19e4f85538, State: Initialized, Role: FOLLOWER
I20260810 06:39:45.351485 20491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7366b68bad4e05ad701f19e4f85538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 06:39:45.351583 20491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:45.351622 20491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:45.351676 20491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:45.352233 20491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7366b68bad4e05ad701f19e4f85538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 06:39:45.352352 20491 leader_election.cc:302] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b7366b68bad4e05ad701f19e4f85538; no voters:
I20260810 06:39:45.352550 20491 leader_election.cc:288] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:45.352610 20497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:45.352831 20491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:45.353005 20497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [term 1 LEADER]: Becoming Leader. State: Replica: 2b7366b68bad4e05ad701f19e4f85538, State: Running, Role: LEADER
I20260810 06:39:45.353112 20497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7366b68bad4e05ad701f19e4f85538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 06:39:45.354151 20500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b7366b68bad4e05ad701f19e4f85538. Latest consensus state: current_term: 1 leader_uuid: "2b7366b68bad4e05ad701f19e4f85538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7366b68bad4e05ad701f19e4f85538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } } }
I20260810 06:39:45.354382 20500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:45.354249 20498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b7366b68bad4e05ad701f19e4f85538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7366b68bad4e05ad701f19e4f85538" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } } }
I20260810 06:39:45.354579 20498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538 [sys.catalog]: This master's current role is: LEADER
W20260810 06:39:45.354820 20512 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:39:45.354871 20512 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:39:45.354962 20513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:45.355219 20513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:45.356073 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:45.344624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57717 (local address 127.0.0.1:44701)
0810 06:39:45.344832 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:45.344842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:45.344877 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:39:45.345547 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:45.345557 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:45.345627 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:45.345784 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:45.346414 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:45.347808 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:45.348616 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:45.349149 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:45.349265 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:45.351791 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:45.351802 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:45.351802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:45.351812 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:45.353990 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:45.354441 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:45.354445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:45.354445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:45.354573 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:45.354832 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:45.354833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:45.354833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:45.355190 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:45.355718 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:45.355805 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 06:39:45.355907 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260810 06:39:45.357012 20513 catalog_manager.cc:1347] Generated new cluster ID: b2e31336c97148079999dbb0c150d3f5
I20260810 06:39:45.357062 20513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:45.357141 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:45.366401 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:45.358719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57467 (local address 127.0.0.1:44701)
0810 06:39:45.358756 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:45.358758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:45.358766 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:39:45.358796 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:45.358797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:45.358806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:45.358866 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:45.359491 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:45.360614 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:45.361590 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:45.361937 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:45.362015 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:45.363423 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:45.363424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:45.363425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:45.363432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:45.365092 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:45.365525 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:45.365527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:45.365527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:45.365592 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:45.365838 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:45.365839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:45.365839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:45.365977 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:45.366085 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:45.366233 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 06:39:45.366305 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:39:45.366962 20513 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:45.367522 20513 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:45.370869 20513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b7366b68bad4e05ad701f19e4f85538: Generated new TSK 0
I20260810 06:39:45.371070 20513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:45.377658 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 20417
W20260810 06:39:45.384342 20400 connection.cc:535] client connection to 127.0.0.1:44701 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44701): Connection reset by peer (error 104)
2026-08-10T06:39:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:45 dist-test-slave-w206 krb5kdc[20524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:45 dist-test-slave-w206 krb5kdc[20524](info): set up 2 sockets
Aug 10 06:39:45 dist-test-slave-w206 krb5kdc[20524](info): commencing operation
krb5kdc: starting...
W20260810 06:39:46.499794 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:46 dist-test-slave-w206 krb5kdc[20524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:46.631246 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:46.713459 20539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:46.713709 20539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:46.713747 20539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:46.714919 20539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:46.715013 20539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:46.715039 20539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:46.715062 20539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:46.715090 20539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:46.716547 20539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:46.716825 20539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:46.717051 20539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:46.722121 20544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:46.722157 20539 server_base.cc:1034] running on GCE node
W20260810 06:39:46.722157 20546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:46.722121 20549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:46.722581 20539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:46.722872 20539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:46.724031 20539 hybrid_clock.cc:648] HybridClock initialized: now 1786343986724038 us; error 45 us; skew 500 ppm
Aug 10 06:39:46 dist-test-slave-w206 krb5kdc[20524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:46.726583 20539 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:46.727175 20539 webserver.cc:460] Webserver started at http://127.0.0.1:33209/ using document root <none> and password file <none>
I20260810 06:39:46.727425 20539 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:46.727496 20539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:46.727648 20539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:46.728720 20539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "7db9655c28ef495e9c0de77ac7ad4f25"
format_stamp: "Formatted at 2026-08-10 06:39:46 on dist-test-slave-w206"
server_key: "fcc5ba2eccb517de88f84959bbf23a1f"
server_key_iv: "f309c1415c6213d957a25d5f5ea9134f"
server_key_version: "encryptionkey@0"
I20260810 06:39:46.729097 20539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "7db9655c28ef495e9c0de77ac7ad4f25"
format_stamp: "Formatted at 2026-08-10 06:39:46 on dist-test-slave-w206"
server_key: "fcc5ba2eccb517de88f84959bbf23a1f"
server_key_iv: "f309c1415c6213d957a25d5f5ea9134f"
server_key_version: "encryptionkey@0"
I20260810 06:39:46.730542 20539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:39:46.731396 20556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:46.731585 20539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:39:46.731694 20539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "7db9655c28ef495e9c0de77ac7ad4f25"
format_stamp: "Formatted at 2026-08-10 06:39:46 on dist-test-slave-w206"
server_key: "fcc5ba2eccb517de88f84959bbf23a1f"
server_key_iv: "f309c1415c6213d957a25d5f5ea9134f"
server_key_version: "encryptionkey@0"
I20260810 06:39:46.731771 20539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:46.752481 20539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:46.754655 20539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:46.754832 20539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:46.758936 20539 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34553
I20260810 06:39:46.759004 20610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34553 every 8 connection(s)
I20260810 06:39:46.759392 20539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:46.760290 20611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:46.762531 20611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25: Bootstrap starting.
I20260810 06:39:46.763077 20611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:46.763320 20611 log.cc:824] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:46.763979 20611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25: No bootstrap required, opened a new log
I20260810 06:39:46.765571 20611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260810 06:39:46.765712 20611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:46.765734 20611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db9655c28ef495e9c0de77ac7ad4f25, State: Initialized, Role: FOLLOWER
I20260810 06:39:46.765846 20611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260810 06:39:46.765900 20611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:46.765923 20611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:46.765997 20611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:46.766551 20611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260810 06:39:46.766676 20611 leader_election.cc:302] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7db9655c28ef495e9c0de77ac7ad4f25; no voters:
I20260810 06:39:46.766703 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 20539
I20260810 06:39:46.766837 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:46.766897 20611 leader_election.cc:288] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:46.766923 20614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:46.767107 31004 external_mini_cluster.cc:1428] Setting key d6ef9004e69f3df4a2d2637391d81035
I20260810 06:39:46.767107 20614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [term 1 LEADER]: Becoming Leader. State: Replica: 7db9655c28ef495e9c0de77ac7ad4f25, State: Running, Role: LEADER
I20260810 06:39:46.767194 20614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260810 06:39:46.767225 20611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:46.767506 20614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7db9655c28ef495e9c0de77ac7ad4f25. Latest consensus state: current_term: 1 leader_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } } }
I20260810 06:39:46.767566 20614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:46.767902 20616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db9655c28ef495e9c0de77ac7ad4f25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } } }
I20260810 06:39:46.768044 20616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:46.768555 20620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:46.769205 20620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:46.770854 20620 catalog_manager.cc:1347] Generated new cluster ID: 5284ab439d634100b8272d85d3284190
I20260810 06:39:46.770907 20620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:46.773214 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:46.767590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53213 (local address 127.0.0.1:34553)
0810 06:39:46.767834 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:46.767844 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:46.768008 (+ 164us) server_negotiation.cc:407] Connection header received
0810 06:39:46.768232 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:46.768238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:46.768355 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:46.768474 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:46.768986 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:46.770096 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:46.771216 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:46.771638 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:46.771766 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:46.772522 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:46.772526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:46.772527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:46.772717 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:46.772748 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:46.772759 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:46.772854 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:46.772932 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 06:39:46.773055 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":85,"threads_started":1}
I20260810 06:39:46.774135 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:46.782111 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:46.775775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54581 (local address 127.0.0.1:34553)
0810 06:39:46.775832 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:46.775834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:46.779276 (+ 3442us) server_negotiation.cc:407] Connection header received
0810 06:39:46.779363 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:46.779364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:46.779375 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:46.779417 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:46.779571 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:46.780079 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:46.780835 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:46.781057 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:46.781133 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:46.781851 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:46.781852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:46.781853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:46.781857 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:46.781871 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:46.781876 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:46.781928 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:46.781988 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 06:39:46.782001 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:39:46.792906 20620 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:46.793470 20620 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:46.799525 20620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7db9655c28ef495e9c0de77ac7ad4f25: Generated new TSK 0
I20260810 06:39:46.799775 20620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:46.811496 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 20539
W20260810 06:39:46.818559 20522 connection.cc:535] client connection to 127.0.0.1:34553 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34553): Connection reset by peer (error 104)
2026-08-10T06:39:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:46 dist-test-slave-w206 krb5kdc[20648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:46 dist-test-slave-w206 krb5kdc[20648](info): set up 2 sockets
Aug 10 06:39:46 dist-test-slave-w206 krb5kdc[20648](info): commencing operation
krb5kdc: starting...
W20260810 06:39:47.918645 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:48 dist-test-slave-w206 krb5kdc[20648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:48.046162 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45023
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:48.130147 20662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:48.130411 20662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:48.130452 20662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:48.131753 20662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:48.131820 20662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:48.131846 20662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:48.131875 20662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:48.131901 20662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:48.133450 20662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45023
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:48.133728 20662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:48.133962 20662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:48.138957 20670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:48.139025 20667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:48.138988 20668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:48.139192 20662 server_base.cc:1034] running on GCE node
I20260810 06:39:48.139492 20662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:48.139804 20662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:48.140978 20662 hybrid_clock.cc:648] HybridClock initialized: now 1786343988140959 us; error 66 us; skew 500 ppm
Aug 10 06:39:48 dist-test-slave-w206 krb5kdc[20648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343988, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:48.143805 20662 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:48.144333 20662 webserver.cc:460] Webserver started at http://127.0.0.1:39667/ using document root <none> and password file <none>
I20260810 06:39:48.144572 20662 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:48.144641 20662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:48.144809 20662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:48.145925 20662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "99146684149941a883954241480918f2"
format_stamp: "Formatted at 2026-08-10 06:39:48 on dist-test-slave-w206"
server_key: "9b76618006b5d5f4cb4a2cc8979c9a24"
server_key_iv: "bbd2b1bec8319eb1bd236ef8d8be0e9b"
server_key_version: "encryptionkey@0"
I20260810 06:39:48.146301 20662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "99146684149941a883954241480918f2"
format_stamp: "Formatted at 2026-08-10 06:39:48 on dist-test-slave-w206"
server_key: "9b76618006b5d5f4cb4a2cc8979c9a24"
server_key_iv: "bbd2b1bec8319eb1bd236ef8d8be0e9b"
server_key_version: "encryptionkey@0"
I20260810 06:39:48.147796 20662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:48.148562 20679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:48.148814 20662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:48.148927 20662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "99146684149941a883954241480918f2"
format_stamp: "Formatted at 2026-08-10 06:39:48 on dist-test-slave-w206"
server_key: "9b76618006b5d5f4cb4a2cc8979c9a24"
server_key_iv: "bbd2b1bec8319eb1bd236ef8d8be0e9b"
server_key_version: "encryptionkey@0"
I20260810 06:39:48.149004 20662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:48.152529 20662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:48.154704 20662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:48.154889 20662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:48.159032 20662 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45023
I20260810 06:39:48.159080 20733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45023 every 8 connection(s)
I20260810 06:39:48.159453 20662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:48.160476 20734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:48.161618 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 20662
I20260810 06:39:48.161746 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:48.161955 31004 external_mini_cluster.cc:1428] Setting key b15c4baa2c9fffdee16006e2bdb6b00e
I20260810 06:39:48.163544 20734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2: Bootstrap starting.
I20260810 06:39:48.164361 20734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:48.164695 20734 log.cc:824] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:48.164840 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:48.162562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52189 (local address 127.0.0.1:45023)
0810 06:39:48.162736 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:48.162750 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:48.162787 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:39:48.163287 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:48.163294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:48.163320 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:48.163440 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:48.164271 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:48.164307 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:48.164307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:48.164312 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:48.164351 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:48.164361 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:48.164473 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:48.164536 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:39:48.164590 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260810 06:39:48.165580 20734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2: No bootstrap required, opened a new log
I20260810 06:39:48.165944 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:48.168387 20734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99146684149941a883954241480918f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260810 06:39:48.168605 20734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:48.168651 20734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99146684149941a883954241480918f2, State: Initialized, Role: FOLLOWER
I20260810 06:39:48.168730 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:48.167664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34799 (local address 127.0.0.1:45023)
0810 06:39:48.167707 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:48.167709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:48.167836 (+ 127us) server_negotiation.cc:407] Connection header received
0810 06:39:48.167901 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:48.167901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:48.167905 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:48.167951 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:48.168486 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:48.168487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:48.168487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:48.168491 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:48.168503 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:48.168506 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:48.168543 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:48.168625 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:39:48.168639 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:39:48.168795 20734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99146684149941a883954241480918f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260810 06:39:48.168882 20734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:48.168905 20734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:48.168934 20734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:48.169565 20734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99146684149941a883954241480918f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260810 06:39:48.169678 20734 leader_election.cc:302] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99146684149941a883954241480918f2; no voters:
I20260810 06:39:48.169842 20734 leader_election.cc:288] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:48.169909 20744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:48.170063 20744 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [term 1 LEADER]: Becoming Leader. State: Replica: 99146684149941a883954241480918f2, State: Running, Role: LEADER
I20260810 06:39:48.170101 20734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:48.170176 20744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99146684149941a883954241480918f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260810 06:39:48.170642 20746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99146684149941a883954241480918f2. Latest consensus state: current_term: 1 leader_uuid: "99146684149941a883954241480918f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99146684149941a883954241480918f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } } }
I20260810 06:39:48.170754 20746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:48.170993 20752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:48.171020 20745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99146684149941a883954241480918f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99146684149941a883954241480918f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } } }
I20260810 06:39:48.171308 20745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:48.171541 20752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:48.173103 20752 catalog_manager.cc:1347] Generated new cluster ID: 65cc57634e76427e96a3213bff795543
I20260810 06:39:48.173156 20752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:48.181241 20752 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:48.181484 20752 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:48.190387 20752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99146684149941a883954241480918f2: Generated new TSK 0
I20260810 06:39:48.190605 20752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:48.194450 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 20662
2026-08-10T06:39:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:48 dist-test-slave-w206 krb5kdc[20765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:48 dist-test-slave-w206 krb5kdc[20765](info): set up 2 sockets
Aug 10 06:39:48 dist-test-slave-w206 krb5kdc[20765](info): commencing operation
krb5kdc: starting...
W20260810 06:39:49.330819 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:49 dist-test-slave-w206 krb5kdc[20765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:49.458379 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:49.543314 20782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:49.543531 20782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:49.543555 20782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:49.544644 20782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:49.544679 20782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:49.544694 20782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:49.544706 20782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:49.544718 20782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:49.546130 20782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:49.546326 20782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:49.546509 20782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:49.551450 20789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:49.551625 20782 server_base.cc:1034] running on GCE node
W20260810 06:39:49.551545 20792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:49.551435 20788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:49.551955 20782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:49.552266 20782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:49.553453 20782 hybrid_clock.cc:648] HybridClock initialized: now 1786343989553431 us; error 40 us; skew 500 ppm
Aug 10 06:39:49 dist-test-slave-w206 krb5kdc[20765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:49.556133 20782 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:49.556617 20782 webserver.cc:460] Webserver started at http://127.0.0.1:45491/ using document root <none> and password file <none>
I20260810 06:39:49.556870 20782 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:49.556939 20782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:49.557083 20782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:49.558162 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "a686d79c3a634463937bf536fe505c62"
format_stamp: "Formatted at 2026-08-10 06:39:49 on dist-test-slave-w206"
server_key: "9804f0f1c4f93c211d8c3e0d7a5983f6"
server_key_iv: "5174f798c4ffd8d89d5552fd8d5aa012"
server_key_version: "encryptionkey@0"
I20260810 06:39:49.558547 20782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "a686d79c3a634463937bf536fe505c62"
format_stamp: "Formatted at 2026-08-10 06:39:49 on dist-test-slave-w206"
server_key: "9804f0f1c4f93c211d8c3e0d7a5983f6"
server_key_iv: "5174f798c4ffd8d89d5552fd8d5aa012"
server_key_version: "encryptionkey@0"
I20260810 06:39:49.560034 20782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:49.560875 20799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:49.561072 20782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:39:49.561182 20782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "a686d79c3a634463937bf536fe505c62"
format_stamp: "Formatted at 2026-08-10 06:39:49 on dist-test-slave-w206"
server_key: "9804f0f1c4f93c211d8c3e0d7a5983f6"
server_key_iv: "5174f798c4ffd8d89d5552fd8d5aa012"
server_key_version: "encryptionkey@0"
I20260810 06:39:49.561257 20782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:49.570525 20782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:49.572948 20782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:49.573123 20782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:49.577304 20782 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43323
I20260810 06:39:49.577332 20854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43323 every 8 connection(s)
I20260810 06:39:49.577705 20782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:49.578735 20855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:49.581408 20855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62: Bootstrap starting.
I20260810 06:39:49.582048 20855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:49.582377 20855 log.cc:824] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:49.583238 20855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62: No bootstrap required, opened a new log
I20260810 06:39:49.584000 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 20782
I20260810 06:39:49.584117 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:49.584362 31004 external_mini_cluster.cc:1428] Setting key b22edadbeed3160b37a614275073a9dc
I20260810 06:39:49.585520 20855 raft_consensus.cc:348] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a686d79c3a634463937bf536fe505c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43323 } }
I20260810 06:39:49.585693 20855 raft_consensus.cc:374] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:49.585772 20855 raft_consensus.cc:729] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a686d79c3a634463937bf536fe505c62, State: Initialized, Role: FOLLOWER
I20260810 06:39:49.585944 20855 consensus_queue.cc:260] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a686d79c3a634463937bf536fe505c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43323 } }
I20260810 06:39:49.586042 20855 raft_consensus.cc:388] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:49.586107 20855 raft_consensus.cc:482] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:49.586169 20855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:49.587268 20855 raft_consensus.cc:504] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a686d79c3a634463937bf536fe505c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43323 } }
I20260810 06:39:49.587401 20855 leader_election.cc:302] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a686d79c3a634463937bf536fe505c62; no voters:
I20260810 06:39:49.587592 20855 leader_election.cc:288] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:49.587615 20860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:49.587740 20860 raft_consensus.cc:686] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [term 1 LEADER]: Becoming Leader. State: Replica: a686d79c3a634463937bf536fe505c62, State: Running, Role: LEADER
I20260810 06:39:49.587823 20860 consensus_queue.cc:237] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a686d79c3a634463937bf536fe505c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43323 } }
I20260810 06:39:49.587878 20855 sys_catalog.cc:564] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:49.588167 20861 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a686d79c3a634463937bf536fe505c62. Latest consensus state: current_term: 1 leader_uuid: "a686d79c3a634463937bf536fe505c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a686d79c3a634463937bf536fe505c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43323 } } }
I20260810 06:39:49.588294 20861 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:49.588466 20863 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a686d79c3a634463937bf536fe505c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a686d79c3a634463937bf536fe505c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43323 } } }
I20260810 06:39:49.588526 20863 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:49.588514 20866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:49.589112 20866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:49.591148 20866 catalog_manager.cc:1347] Generated new cluster ID: 6fbc58a2817d4c3da01e555315c15554
I20260810 06:39:49.591197 20866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:39:49 dist-test-slave-w206 krb5kdc[20765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:39:49.596309 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:49.584933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33733 (local address 127.0.0.1:43323)
0810 06:39:49.585144 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:49.585154 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:49.585190 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:39:49.585477 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:49.585485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:49.585549 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:49.585704 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:49.586671 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:49.587798 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:49.588927 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:49.589518 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:49.589624 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:49.592401 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:49.592405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:49.592405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:49.592417 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:49.594426 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:49.594824 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:49.594828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:49.594828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:49.594962 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:49.595187 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:49.595188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:49.595188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:49.595509 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:49.595697 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:49.595770 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 06:39:49.595913 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260810 06:39:49.597471 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:49.601425 20866 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:49.602120 20866 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:49.606204 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:49.598888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47509 (local address 127.0.0.1:43323)
0810 06:39:49.598925 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:49.598927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:49.599121 (+ 194us) server_negotiation.cc:407] Connection header received
0810 06:39:49.599160 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:49.599160 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:49.599172 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:49.599229 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:49.599847 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:49.600582 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:49.601552 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:49.601877 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:49.601926 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:49.603466 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:49.603468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:49.603468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:49.603476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:49.605060 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:49.605429 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:49.605430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:49.605430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:49.605488 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:49.605690 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:49.605691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:49.605692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:49.605819 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:49.605915 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:49.606017 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 06:39:49.606112 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":20}
I20260810 06:39:49.609815 20866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a686d79c3a634463937bf536fe505c62: Generated new TSK 0
I20260810 06:39:49.610004 20866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:49.617614 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 20782
2026-08-10T06:39:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:49 dist-test-slave-w206 krb5kdc[20889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:49 dist-test-slave-w206 krb5kdc[20889](info): set up 2 sockets
Aug 10 06:39:49 dist-test-slave-w206 krb5kdc[20889](info): commencing operation
krb5kdc: starting...
W20260810 06:39:50.719820 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:50 dist-test-slave-w206 krb5kdc[20889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:50.854748 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:50.938866 20902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:50.939179 20902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:50.939222 20902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:50.940416 20902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:50.940490 20902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:50.940514 20902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:50.940547 20902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:50.940575 20902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:50.942078 20902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:50.942364 20902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:50.942606 20902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:39:50.947844 20902 server_base.cc:1034] running on GCE node
W20260810 06:39:50.947782 20907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:50.947786 20908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:50.947764 20911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:50.948397 20902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:50.948706 20902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:50.949875 20902 hybrid_clock.cc:648] HybridClock initialized: now 1786343990949871 us; error 38 us; skew 500 ppm
Aug 10 06:39:50 dist-test-slave-w206 krb5kdc[20889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:50.952639 20902 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:50.953168 20902 webserver.cc:460] Webserver started at http://127.0.0.1:35289/ using document root <none> and password file <none>
I20260810 06:39:50.953469 20902 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:50.953542 20902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:50.953691 20902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:50.954802 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "e4d169db9cf14f388d7c1bcbf3314a5a"
format_stamp: "Formatted at 2026-08-10 06:39:50 on dist-test-slave-w206"
server_key: "5f9aaaa3aced711b81dae90af720360b"
server_key_iv: "a46f69dc68ed25356c328d4fc412c949"
server_key_version: "encryptionkey@0"
I20260810 06:39:50.955199 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "e4d169db9cf14f388d7c1bcbf3314a5a"
format_stamp: "Formatted at 2026-08-10 06:39:50 on dist-test-slave-w206"
server_key: "5f9aaaa3aced711b81dae90af720360b"
server_key_iv: "a46f69dc68ed25356c328d4fc412c949"
server_key_version: "encryptionkey@0"
I20260810 06:39:50.956647 20902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:50.957489 20920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:50.957662 20902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:39:50.957760 20902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "e4d169db9cf14f388d7c1bcbf3314a5a"
format_stamp: "Formatted at 2026-08-10 06:39:50 on dist-test-slave-w206"
server_key: "5f9aaaa3aced711b81dae90af720360b"
server_key_iv: "a46f69dc68ed25356c328d4fc412c949"
server_key_version: "encryptionkey@0"
I20260810 06:39:50.957845 20902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:50.977689 20902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:50.980163 20902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:50.980346 20902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:50.984615 20902 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37743
I20260810 06:39:50.984654 20976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37743 every 8 connection(s)
I20260810 06:39:50.985036 20902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:50.985946 20977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:50.988332 20977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a: Bootstrap starting.
I20260810 06:39:50.988919 20977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:50.989207 20977 log.cc:824] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:50.989976 20977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a: No bootstrap required, opened a new log
I20260810 06:39:50.990598 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 20902
I20260810 06:39:50.990765 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:50.991056 31004 external_mini_cluster.cc:1428] Setting key 75b0808986c75b31abf0c320dd0a1c21
I20260810 06:39:50.992332 20977 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37743 } }
I20260810 06:39:50.992511 20977 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:50.992552 20977 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d169db9cf14f388d7c1bcbf3314a5a, State: Initialized, Role: FOLLOWER
I20260810 06:39:50.992707 20977 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37743 } }
I20260810 06:39:50.992815 20977 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:50.992848 20977 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:50.992888 20977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:50.993655 20977 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37743 } }
I20260810 06:39:50.993765 20977 leader_election.cc:302] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4d169db9cf14f388d7c1bcbf3314a5a; no voters:
I20260810 06:39:50.993919 20977 leader_election.cc:288] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:50.993996 20982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:50.994163 20977 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:50.994194 20982 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [term 1 LEADER]: Becoming Leader. State: Replica: e4d169db9cf14f388d7c1bcbf3314a5a, State: Running, Role: LEADER
I20260810 06:39:50.994418 20982 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37743 } }
I20260810 06:39:50.994894 20983 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37743 } } }
I20260810 06:39:50.994994 20983 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:50.995402 20984 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4d169db9cf14f388d7c1bcbf3314a5a. Latest consensus state: current_term: 1 leader_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d169db9cf14f388d7c1bcbf3314a5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37743 } } }
I20260810 06:39:50.995519 20984 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:50.995682 20991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:50.996224 20991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:50.997952 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:50.991668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44379 (local address 127.0.0.1:37743)
0810 06:39:50.991909 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:50.991919 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:50.991954 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:39:50.992282 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:50.992291 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:50.992353 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:50.992504 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:50.992809 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:50.994521 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:50.995727 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:50.996363 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:50.996505 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:50.997226 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:50.997240 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:50.997240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:50.997460 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:50.997502 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:50.997517 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:50.997585 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:50.997684 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 06:39:50.997751 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":98,"threads_started":1}
I20260810 06:39:50.997968 20991 catalog_manager.cc:1347] Generated new cluster ID: 3899e7b45ed34d04af179cd37568c9d0
I20260810 06:39:50.998034 20991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:50.998999 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:51.004667 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:51.000594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58267 (local address 127.0.0.1:37743)
0810 06:39:51.000628 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:51.000630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:51.000899 (+ 269us) server_negotiation.cc:407] Connection header received
0810 06:39:51.000992 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:51.000993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:51.001015 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:51.001044 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:51.001255 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:51.001952 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:51.003066 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:51.003569 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:51.003617 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:51.004402 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:51.004403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:51.004404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:51.004409 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:51.004423 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:51.004428 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:51.004455 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:51.004562 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 06:39:51.004588 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 06:39:51.010438 20991 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:51.010965 20991 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:51.014640 20991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4d169db9cf14f388d7c1bcbf3314a5a: Generated new TSK 0
I20260810 06:39:51.014832 20991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:51.021885 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 20902
2026-08-10T06:39:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:51 dist-test-slave-w206 krb5kdc[21010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:51 dist-test-slave-w206 krb5kdc[21010](info): set up 2 sockets
Aug 10 06:39:51 dist-test-slave-w206 krb5kdc[21010](info): commencing operation
krb5kdc: starting...
W20260810 06:39:52.147163 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:52 dist-test-slave-w206 krb5kdc[21010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:52.285913 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:52.369693 21025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:52.369928 21025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:52.369952 21025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:52.371080 21025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:52.371119 21025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:52.371133 21025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:52.371145 21025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:52.371157 21025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:52.372576 21025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:52.372778 21025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:52.372961 21025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:52.378000 21031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:52.378007 21033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:52.378006 21030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:52.378252 21025 server_base.cc:1034] running on GCE node
I20260810 06:39:52.378535 21025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:52.378845 21025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:52.379993 21025 hybrid_clock.cc:648] HybridClock initialized: now 1786343992379967 us; error 38 us; skew 500 ppm
Aug 10 06:39:52 dist-test-slave-w206 krb5kdc[21010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343992, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:52.388664 21025 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:52.389262 21025 webserver.cc:460] Webserver started at http://127.0.0.1:41769/ using document root <none> and password file <none>
I20260810 06:39:52.389510 21025 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:52.389583 21025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:52.389750 21025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:52.390883 21025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "803592f8f25243798b367d28d7c8149b"
format_stamp: "Formatted at 2026-08-10 06:39:52 on dist-test-slave-w206"
server_key: "0b11f165e28ca46e30803d8c3547b3bd"
server_key_iv: "3fd25816f20144a6193343652496824e"
server_key_version: "encryptionkey@0"
I20260810 06:39:52.391340 21025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "803592f8f25243798b367d28d7c8149b"
format_stamp: "Formatted at 2026-08-10 06:39:52 on dist-test-slave-w206"
server_key: "0b11f165e28ca46e30803d8c3547b3bd"
server_key_iv: "3fd25816f20144a6193343652496824e"
server_key_version: "encryptionkey@0"
I20260810 06:39:52.392750 21025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:52.393641 21040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:52.393882 21025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:52.393996 21025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "803592f8f25243798b367d28d7c8149b"
format_stamp: "Formatted at 2026-08-10 06:39:52 on dist-test-slave-w206"
server_key: "0b11f165e28ca46e30803d8c3547b3bd"
server_key_iv: "3fd25816f20144a6193343652496824e"
server_key_version: "encryptionkey@0"
I20260810 06:39:52.394074 21025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:52.397593 21025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:52.400082 21025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:52.400249 21025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:52.404515 21025 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45655
I20260810 06:39:52.404567 21101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45655 every 8 connection(s)
I20260810 06:39:52.404945 21025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:52.405992 21102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:52.408438 21102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b: Bootstrap starting.
I20260810 06:39:52.409118 21102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:52.409416 21102 log.cc:824] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:52.410123 21102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b: No bootstrap required, opened a new log
I20260810 06:39:52.411459 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21025
I20260810 06:39:52.411608 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:52.411804 31004 external_mini_cluster.cc:1428] Setting key 213bdb4fc8a68e441aaa17a61f6d9997
I20260810 06:39:52.412039 21102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803592f8f25243798b367d28d7c8149b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260810 06:39:52.412259 21102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:52.412386 21102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 803592f8f25243798b367d28d7c8149b, State: Initialized, Role: FOLLOWER
I20260810 06:39:52.412544 21102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803592f8f25243798b367d28d7c8149b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260810 06:39:52.412642 21102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:52.412667 21102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:52.412741 21102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:52.413873 21102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803592f8f25243798b367d28d7c8149b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260810 06:39:52.413996 21102 leader_election.cc:302] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 803592f8f25243798b367d28d7c8149b; no voters:
I20260810 06:39:52.414249 21107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:52.414227 21102 leader_election.cc:288] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:52.414415 21107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [term 1 LEADER]: Becoming Leader. State: Replica: 803592f8f25243798b367d28d7c8149b, State: Running, Role: LEADER
I20260810 06:39:52.414570 21107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803592f8f25243798b367d28d7c8149b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260810 06:39:52.414620 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:52.412390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48659 (local address 127.0.0.1:45655)
0810 06:39:52.412628 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:52.412656 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:52.412693 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:39:52.412875 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:52.412882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:52.412908 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:52.413054 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:52.413924 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:52.413928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:52.413929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:52.413935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:52.413993 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:52.414009 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:52.414084 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:52.414167 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:39:52.414309 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":104,"threads_started":1}
I20260810 06:39:52.414664 21102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:52.414955 21108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 803592f8f25243798b367d28d7c8149b. Latest consensus state: current_term: 1 leader_uuid: "803592f8f25243798b367d28d7c8149b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803592f8f25243798b367d28d7c8149b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } } }
I20260810 06:39:52.415066 21108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:52.415019 21109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "803592f8f25243798b367d28d7c8149b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803592f8f25243798b367d28d7c8149b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } } }
I20260810 06:39:52.415112 21109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:52.415328 21111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:52.415680 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:52.415998 21111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:52.417721 21111 catalog_manager.cc:1347] Generated new cluster ID: e8c33df51c654dccac76510ccb377a7a
I20260810 06:39:52.417816 21111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:52.418646 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:52.417269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40977 (local address 127.0.0.1:45655)
0810 06:39:52.417306 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:52.417308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:52.417613 (+ 305us) server_negotiation.cc:407] Connection header received
0810 06:39:52.417688 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:52.417689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:52.417706 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:52.417758 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:52.418425 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:52.418426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:52.418426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:52.418431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:52.418446 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:52.418450 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:52.418508 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:52.418567 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 06:39:52.418581 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:39:52.430425 21111 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:52.430656 21111 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:52.434355 21111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 803592f8f25243798b367d28d7c8149b: Generated new TSK 0
I20260810 06:39:52.434554 21111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:52.437873 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21025
2026-08-10T06:39:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:52 dist-test-slave-w206 krb5kdc[21135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:52 dist-test-slave-w206 krb5kdc[21135](info): set up 2 sockets
Aug 10 06:39:52 dist-test-slave-w206 krb5kdc[21135](info): commencing operation
krb5kdc: starting...
W20260810 06:39:53.551434 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:53 dist-test-slave-w206 krb5kdc[21135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:53.687479 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:53.772950 21149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:53.773234 21149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:53.773276 21149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:53.774487 21149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:53.774573 21149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:53.774601 21149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:53.774636 21149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:53.774660 21149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:53.776276 21149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:53.776583 21149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:53.776832 21149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:53.782404 21155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:53.782419 21158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:53.782438 21149 server_base.cc:1034] running on GCE node
W20260810 06:39:53.782404 21156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:53.782953 21149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:53.783365 21149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:53.784556 21149 hybrid_clock.cc:648] HybridClock initialized: now 1786343993784533 us; error 45 us; skew 500 ppm
Aug 10 06:39:53 dist-test-slave-w206 krb5kdc[21135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:53.787391 21149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:53.788043 21149 webserver.cc:460] Webserver started at http://127.0.0.1:41095/ using document root <none> and password file <none>
I20260810 06:39:53.788342 21149 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:53.788425 21149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:53.788594 21149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:53.789860 21149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "2cc1d29917f64276804c8f70dd59e306"
format_stamp: "Formatted at 2026-08-10 06:39:53 on dist-test-slave-w206"
server_key: "2e494474f9114d71fb0bc8f96fbc943c"
server_key_iv: "c30bbb8070f131479705bc9cef57dd34"
server_key_version: "encryptionkey@0"
I20260810 06:39:53.790341 21149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "2cc1d29917f64276804c8f70dd59e306"
format_stamp: "Formatted at 2026-08-10 06:39:53 on dist-test-slave-w206"
server_key: "2e494474f9114d71fb0bc8f96fbc943c"
server_key_iv: "c30bbb8070f131479705bc9cef57dd34"
server_key_version: "encryptionkey@0"
I20260810 06:39:53.792019 21149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260810 06:39:53.793185 21165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:53.793463 21149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:53.793601 21149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "2cc1d29917f64276804c8f70dd59e306"
format_stamp: "Formatted at 2026-08-10 06:39:53 on dist-test-slave-w206"
server_key: "2e494474f9114d71fb0bc8f96fbc943c"
server_key_iv: "c30bbb8070f131479705bc9cef57dd34"
server_key_version: "encryptionkey@0"
I20260810 06:39:53.793716 21149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:53.813737 21149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:53.816110 21149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:53.816236 21149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:53.820722 21149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41725
I20260810 06:39:53.820925 21224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41725 every 8 connection(s)
I20260810 06:39:53.821165 21149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:53.822106 21225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:53.823750 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21149
I20260810 06:39:53.823894 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:53.824097 31004 external_mini_cluster.cc:1428] Setting key 04636e5ed33b675bd121e2d34596be16
I20260810 06:39:53.825127 21225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306: Bootstrap starting.
I20260810 06:39:53.825788 21225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:53.826128 21225 log.cc:824] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:53.827099 21225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306: No bootstrap required, opened a new log
I20260810 06:39:53.829769 21225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc1d29917f64276804c8f70dd59e306" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41725 } }
I20260810 06:39:53.829972 21225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:53.830014 21225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1d29917f64276804c8f70dd59e306, State: Initialized, Role: FOLLOWER
I20260810 06:39:53.830243 21225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc1d29917f64276804c8f70dd59e306" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41725 } }
I20260810 06:39:53.830367 21225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:53.830402 21225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:53.830458 21225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:53.831199 21225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc1d29917f64276804c8f70dd59e306" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41725 } }
I20260810 06:39:53.831341 21225 leader_election.cc:302] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cc1d29917f64276804c8f70dd59e306; no voters:
I20260810 06:39:53.831549 21225 leader_election.cc:288] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:53.831593 21230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:53.831806 21230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [term 1 LEADER]: Becoming Leader. State: Replica: 2cc1d29917f64276804c8f70dd59e306, State: Running, Role: LEADER
I20260810 06:39:53.831868 21225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:53.831995 21230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc1d29917f64276804c8f70dd59e306" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41725 } }
Aug 10 06:39:53 dist-test-slave-w206 krb5kdc[21135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:39:53.832433 21230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cc1d29917f64276804c8f70dd59e306. Latest consensus state: current_term: 1 leader_uuid: "2cc1d29917f64276804c8f70dd59e306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc1d29917f64276804c8f70dd59e306" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41725 } } }
I20260810 06:39:53.832445 21232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cc1d29917f64276804c8f70dd59e306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc1d29917f64276804c8f70dd59e306" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41725 } } }
I20260810 06:39:53.832598 21232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:53.832818 21230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:53.833001 21239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:53.834254 21239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:53.836205 21239 catalog_manager.cc:1347] Generated new cluster ID: 4e712668280a40059c1423af19c11efe
I20260810 06:39:53.836261 21239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:53.837497 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:53.824842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:35355 (local address 127.0.0.1:41725)
0810 06:39:53.825056 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:53.825066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:53.825103 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:39:53.825579 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:53.825585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:53.825641 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:53.825798 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:53.826611 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:53.828256 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:53.829068 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:53.829829 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:53.829957 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:53.833052 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:53.833056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:53.833056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:53.833068 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:53.835511 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:53.836071 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:53.836075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:53.836075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:53.836189 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:53.836505 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:53.836507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:53.836507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:53.836823 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:53.836970 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:53.837165 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 06:39:53.837279 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260810 06:39:53.838629 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:53.848066 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:53.840311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:35303 (local address 127.0.0.1:41725)
0810 06:39:53.840346 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:53.840348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:53.840356 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:39:53.840411 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:53.840412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:53.840421 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:53.840476 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:53.841192 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:53.841997 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:53.843124 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:53.843490 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:53.843559 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:53.845058 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:53.845059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:53.845060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:53.845067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:53.846708 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:53.847172 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:53.847173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:53.847173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:53.847243 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:53.847480 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:53.847480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:53.847481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:53.847612 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:53.847733 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:53.847897 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:39:53.847966 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:39:53.874466 21239 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:53.875336 21239 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:53.881440 21239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cc1d29917f64276804c8f70dd59e306: Generated new TSK 0
I20260810 06:39:53.881673 21239 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:53.886080 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21149
W20260810 06:39:53.891517 21133 connection.cc:535] client connection to 127.0.0.1:41725 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41725): Connection reset by peer (error 104)
2026-08-10T06:39:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:53 dist-test-slave-w206 krb5kdc[21256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:53 dist-test-slave-w206 krb5kdc[21256](info): set up 2 sockets
Aug 10 06:39:53 dist-test-slave-w206 krb5kdc[21256](info): commencing operation
krb5kdc: starting...
W20260810 06:39:54.972601 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:55 dist-test-slave-w206 krb5kdc[21256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:55.105885 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:55.190451 21270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:55.190728 21270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:55.190770 21270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:55.192045 21270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:55.192127 21270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:55.192152 21270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:55.192174 21270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:55.192204 21270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:55.194350 21270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:55.194644 21270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:55.194895 21270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:55.200667 21276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:55.200623 21278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:55.200647 21275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:55.200827 21270 server_base.cc:1034] running on GCE node
I20260810 06:39:55.201160 21270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:55.201447 21270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:55.202610 21270 hybrid_clock.cc:648] HybridClock initialized: now 1786343995202595 us; error 32 us; skew 500 ppm
Aug 10 06:39:55 dist-test-slave-w206 krb5kdc[21256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:55.205700 21270 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:55.206220 21270 webserver.cc:460] Webserver started at http://127.0.0.1:45587/ using document root <none> and password file <none>
I20260810 06:39:55.206455 21270 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:55.206504 21270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:55.206604 21270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:55.207799 21270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "a978248a50304538a26cffabb860f643"
format_stamp: "Formatted at 2026-08-10 06:39:55 on dist-test-slave-w206"
server_key: "92dfc988b44a848193f102c3ca4630db"
server_key_iv: "86cca361275f920d524c7446226d3f36"
server_key_version: "encryptionkey@0"
I20260810 06:39:55.208125 21270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "a978248a50304538a26cffabb860f643"
format_stamp: "Formatted at 2026-08-10 06:39:55 on dist-test-slave-w206"
server_key: "92dfc988b44a848193f102c3ca4630db"
server_key_iv: "86cca361275f920d524c7446226d3f36"
server_key_version: "encryptionkey@0"
I20260810 06:39:55.209589 21270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:39:55.210412 21286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:55.210655 21270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:55.210774 21270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "a978248a50304538a26cffabb860f643"
format_stamp: "Formatted at 2026-08-10 06:39:55 on dist-test-slave-w206"
server_key: "92dfc988b44a848193f102c3ca4630db"
server_key_iv: "86cca361275f920d524c7446226d3f36"
server_key_version: "encryptionkey@0"
I20260810 06:39:55.210861 21270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:55.223699 21270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:55.226176 21270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:55.226370 21270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:55.230842 21270 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43005
I20260810 06:39:55.230892 21344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43005 every 8 connection(s)
I20260810 06:39:55.231365 21270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:55.232031 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21270
I20260810 06:39:55.232235 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:55.232498 31004 external_mini_cluster.cc:1428] Setting key b8f5e3a29e60aeabb9db28e9e06c1af1
I20260810 06:39:55.232656 21345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:55.236393 21345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643: Bootstrap starting.
I20260810 06:39:55.237140 21345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:55.237502 21345 log.cc:824] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:55.238420 21345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643: No bootstrap required, opened a new log
I20260810 06:39:55.238670 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:55.233089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:59259 (local address 127.0.0.1:43005)
0810 06:39:55.233335 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:55.233343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:55.233371 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:39:55.233906 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:55.233913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:55.233962 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:55.234077 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:55.234337 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:55.235320 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:55.236535 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:55.236973 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:55.237062 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:55.237921 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:55.237926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:55.237926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:55.238128 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:55.238161 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:55.238172 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:55.238241 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:55.238420 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 06:39:55.238456 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260810 06:39:55.239787 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:55.241495 21345 raft_consensus.cc:348] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a978248a50304538a26cffabb860f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43005 } }
I20260810 06:39:55.241737 21345 raft_consensus.cc:374] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:55.241786 21345 raft_consensus.cc:729] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a978248a50304538a26cffabb860f643, State: Initialized, Role: FOLLOWER
I20260810 06:39:55.241922 21345 consensus_queue.cc:260] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a978248a50304538a26cffabb860f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43005 } }
I20260810 06:39:55.242022 21345 raft_consensus.cc:388] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:55.242092 21345 raft_consensus.cc:482] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:55.242154 21345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:55.243026 21345 raft_consensus.cc:504] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a978248a50304538a26cffabb860f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43005 } }
I20260810 06:39:55.243156 21345 leader_election.cc:302] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a978248a50304538a26cffabb860f643; no voters:
I20260810 06:39:55.243345 21345 leader_election.cc:288] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:55.243397 21357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:55.243548 21357 raft_consensus.cc:686] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [term 1 LEADER]: Becoming Leader. State: Replica: a978248a50304538a26cffabb860f643, State: Running, Role: LEADER
I20260810 06:39:55.243650 21345 sys_catalog.cc:564] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:55.243705 21357 consensus_queue.cc:237] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a978248a50304538a26cffabb860f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43005 } }
I20260810 06:39:55.244072 21357 sys_catalog.cc:455] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a978248a50304538a26cffabb860f643. Latest consensus state: current_term: 1 leader_uuid: "a978248a50304538a26cffabb860f643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a978248a50304538a26cffabb860f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43005 } } }
I20260810 06:39:55.244089 21360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a978248a50304538a26cffabb860f643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a978248a50304538a26cffabb860f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43005 } } }
I20260810 06:39:55.244225 21357 sys_catalog.cc:458] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:55.244302 21360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:55.244603 21365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:55.245270 21365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:55.245455 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:55.241593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:39203 (local address 127.0.0.1:43005)
0810 06:39:55.241634 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:55.241637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:55.241709 (+ 72us) server_negotiation.cc:407] Connection header received
0810 06:39:55.241792 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:55.241793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:55.241805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:55.241875 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:55.242082 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:55.242871 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:55.243940 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:55.244230 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:55.244289 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:55.245121 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:55.245122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:55.245122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:55.245128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:55.245143 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:55.245147 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:55.245197 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:55.245368 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 06:39:55.245380 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":20}
I20260810 06:39:55.247308 21365 catalog_manager.cc:1347] Generated new cluster ID: 3ae5fec1bc074485886754754ad52884
I20260810 06:39:55.247365 21365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:55.281404 21365 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:55.282050 21365 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:55.290138 21365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a978248a50304538a26cffabb860f643: Generated new TSK 0
I20260810 06:39:55.290380 21365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:55.296293 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21270
W20260810 06:39:55.303488 21254 connection.cc:535] client connection to 127.0.0.1:43005 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43005): Connection reset by peer (error 104)
2026-08-10T06:39:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:55 dist-test-slave-w206 krb5kdc[21380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:55 dist-test-slave-w206 krb5kdc[21380](info): set up 2 sockets
Aug 10 06:39:55 dist-test-slave-w206 krb5kdc[21380](info): commencing operation
krb5kdc: starting...
W20260810 06:39:56.411772 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:56 dist-test-slave-w206 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:56.558614 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43083
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:56.642915 21394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:56.643175 21394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:56.643200 21394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:56.644304 21394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:56.644352 21394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:56.644367 21394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:56.644380 21394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:56.644392 21394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:56.645821 21394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:56.646025 21394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:56.646196 21394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:39:56.651297 21400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:56.651383 21403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:56.651360 21394 server_base.cc:1034] running on GCE node
W20260810 06:39:56.651361 21401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:56.651775 21394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:56.652045 21394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:56.653398 21394 hybrid_clock.cc:648] HybridClock initialized: now 1786343996653375 us; error 30 us; skew 500 ppm
Aug 10 06:39:56 dist-test-slave-w206 krb5kdc[21380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343996, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:56.656086 21394 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:56.656626 21394 webserver.cc:460] Webserver started at http://127.0.0.1:38565/ using document root <none> and password file <none>
I20260810 06:39:56.656895 21394 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:56.656944 21394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:56.657119 21394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:56.658221 21394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "017aa7bf4921453fb7a63a483bab022e"
format_stamp: "Formatted at 2026-08-10 06:39:56 on dist-test-slave-w206"
server_key: "2da35c61a28ff93a9a5748b47f77e600"
server_key_iv: "317f3975cfee288d6693454eb5acfa0e"
server_key_version: "encryptionkey@0"
I20260810 06:39:56.658597 21394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "017aa7bf4921453fb7a63a483bab022e"
format_stamp: "Formatted at 2026-08-10 06:39:56 on dist-test-slave-w206"
server_key: "2da35c61a28ff93a9a5748b47f77e600"
server_key_iv: "317f3975cfee288d6693454eb5acfa0e"
server_key_version: "encryptionkey@0"
I20260810 06:39:56.660154 21394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:56.661001 21411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:56.661245 21394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 06:39:56.661341 21394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "017aa7bf4921453fb7a63a483bab022e"
format_stamp: "Formatted at 2026-08-10 06:39:56 on dist-test-slave-w206"
server_key: "2da35c61a28ff93a9a5748b47f77e600"
server_key_iv: "317f3975cfee288d6693454eb5acfa0e"
server_key_version: "encryptionkey@0"
I20260810 06:39:56.661419 21394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:56.665040 21394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:56.667528 21394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:56.667692 21394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:56.672186 21394 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43083
I20260810 06:39:56.672401 21467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43083 every 8 connection(s)
I20260810 06:39:56.672624 21394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:56.673646 21468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:56.674096 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21394
I20260810 06:39:56.674238 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:56.674474 31004 external_mini_cluster.cc:1428] Setting key 0789764b88a5d310b07d629e555dcc2a
I20260810 06:39:56.676801 21468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e: Bootstrap starting.
I20260810 06:39:56.677027 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:56.675061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:47411 (local address 127.0.0.1:43083)
0810 06:39:56.675247 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:56.675255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:56.675282 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:39:56.675675 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:56.675682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:56.675706 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:56.675844 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:56.676523 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:56.676526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:56.676526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:56.676530 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:56.676566 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:56.676576 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:56.676634 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:56.676724 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 06:39:56.676802 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260810 06:39:56.677546 21468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:56.677845 21468 log.cc:824] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:56.677879 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:56.678560 21468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e: No bootstrap required, opened a new log
I20260810 06:39:56.680321 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:56.679345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:48475 (local address 127.0.0.1:43083)
0810 06:39:56.679384 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:56.679386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:56.679455 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:39:56.679525 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:56.679525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:56.679529 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:56.679569 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:56.680114 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:56.680115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:56.680115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:56.680118 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:56.680129 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:56.680132 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:56.680174 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:56.680228 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 06:39:56.680238 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:39:56.681162 21468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017aa7bf4921453fb7a63a483bab022e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43083 } }
I20260810 06:39:56.681327 21468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:56.681378 21468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017aa7bf4921453fb7a63a483bab022e, State: Initialized, Role: FOLLOWER
I20260810 06:39:56.681506 21468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017aa7bf4921453fb7a63a483bab022e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43083 } }
I20260810 06:39:56.681602 21468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:56.681646 21468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:56.681696 21468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:56.682268 21468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017aa7bf4921453fb7a63a483bab022e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43083 } }
I20260810 06:39:56.682443 21468 leader_election.cc:302] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 017aa7bf4921453fb7a63a483bab022e; no voters:
I20260810 06:39:56.682646 21468 leader_election.cc:288] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:56.682708 21480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:56.682888 21480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [term 1 LEADER]: Becoming Leader. State: Replica: 017aa7bf4921453fb7a63a483bab022e, State: Running, Role: LEADER
I20260810 06:39:56.682932 21468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:56.683171 21480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017aa7bf4921453fb7a63a483bab022e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43083 } }
I20260810 06:39:56.683622 21481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "017aa7bf4921453fb7a63a483bab022e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017aa7bf4921453fb7a63a483bab022e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43083 } } }
I20260810 06:39:56.683657 21482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 017aa7bf4921453fb7a63a483bab022e. Latest consensus state: current_term: 1 leader_uuid: "017aa7bf4921453fb7a63a483bab022e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017aa7bf4921453fb7a63a483bab022e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43083 } } }
I20260810 06:39:56.683750 21481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:56.683759 21482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:56.683962 21491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:56.684512 21491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:56.686106 21491 catalog_manager.cc:1347] Generated new cluster ID: c75f0c4af88b498291c0b2ea94afe255
I20260810 06:39:56.686175 21491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:56.700116 21491 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:56.700348 21491 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:56.706913 21491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 017aa7bf4921453fb7a63a483bab022e: Generated new TSK 0
I20260810 06:39:56.707154 21491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:56.710546 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21394
2026-08-10T06:39:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:56 dist-test-slave-w206 krb5kdc[21504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:56 dist-test-slave-w206 krb5kdc[21504](info): set up 2 sockets
Aug 10 06:39:56 dist-test-slave-w206 krb5kdc[21504](info): commencing operation
krb5kdc: starting...
W20260810 06:39:57.896070 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:57 dist-test-slave-w206 krb5kdc[21504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:58.022652 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:58.106895 21519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:58.107196 21519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:58.107221 21519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:58.108404 21519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:58.108449 21519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:58.108464 21519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:58.108476 21519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:58.108489 21519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:58.110021 21519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46797
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:58.110294 21519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:58.110543 21519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:39:58.115701 21519 server_base.cc:1034] running on GCE node
W20260810 06:39:58.115795 21526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:58.115670 21527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:58.115684 21529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:58.116137 21519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:58.116469 21519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:58.117625 21519 hybrid_clock.cc:648] HybridClock initialized: now 1786343998117604 us; error 39 us; skew 500 ppm
Aug 10 06:39:58 dist-test-slave-w206 krb5kdc[21504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:58.124547 21519 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:58.125015 21519 webserver.cc:460] Webserver started at http://127.0.0.1:42657/ using document root <none> and password file <none>
I20260810 06:39:58.125284 21519 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:58.125334 21519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:58.125469 21519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:58.126677 21519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "b487477607344f44a652cad9345f2c7d"
format_stamp: "Formatted at 2026-08-10 06:39:58 on dist-test-slave-w206"
server_key: "644d5c48965b152f69cb4814c05c750a"
server_key_iv: "40c07e7cbbf14d9b04940009b551d9e0"
server_key_version: "encryptionkey@0"
I20260810 06:39:58.127094 21519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "b487477607344f44a652cad9345f2c7d"
format_stamp: "Formatted at 2026-08-10 06:39:58 on dist-test-slave-w206"
server_key: "644d5c48965b152f69cb4814c05c750a"
server_key_iv: "40c07e7cbbf14d9b04940009b551d9e0"
server_key_version: "encryptionkey@0"
I20260810 06:39:58.128558 21519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:58.129437 21536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:58.129757 21519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:39:58.129861 21519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "b487477607344f44a652cad9345f2c7d"
format_stamp: "Formatted at 2026-08-10 06:39:58 on dist-test-slave-w206"
server_key: "644d5c48965b152f69cb4814c05c750a"
server_key_iv: "40c07e7cbbf14d9b04940009b551d9e0"
server_key_version: "encryptionkey@0"
I20260810 06:39:58.129940 21519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:58.140875 21519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:58.143441 21519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:58.143608 21519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:58.147617 21519 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46797
I20260810 06:39:58.147673 21595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46797 every 8 connection(s)
I20260810 06:39:58.148033 21519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:58.148185 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21519
I20260810 06:39:58.148337 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:58.148583 31004 external_mini_cluster.cc:1428] Setting key 4e677662bc713f0543e1623eea765f20
I20260810 06:39:58.149166 21596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:58.152604 21596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d: Bootstrap starting.
I20260810 06:39:58.153338 21596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:58.153713 21596 log.cc:824] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:58.154541 21596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d: No bootstrap required, opened a new log
Aug 10 06:39:58 dist-test-slave-w206 krb5kdc[21504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:39:58.156961 21596 raft_consensus.cc:348] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487477607344f44a652cad9345f2c7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260810 06:39:58.157178 21596 raft_consensus.cc:374] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:58.157238 21596 raft_consensus.cc:729] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b487477607344f44a652cad9345f2c7d, State: Initialized, Role: FOLLOWER
I20260810 06:39:58.157385 21596 consensus_queue.cc:260] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487477607344f44a652cad9345f2c7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260810 06:39:58.157496 21596 raft_consensus.cc:388] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:58.157532 21596 raft_consensus.cc:482] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:58.157584 21596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:58.158262 21596 raft_consensus.cc:504] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487477607344f44a652cad9345f2c7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260810 06:39:58.158396 21596 leader_election.cc:302] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b487477607344f44a652cad9345f2c7d; no voters:
I20260810 06:39:58.158618 21596 leader_election.cc:288] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:58.158703 21601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:58.158916 21601 raft_consensus.cc:686] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [term 1 LEADER]: Becoming Leader. State: Replica: b487477607344f44a652cad9345f2c7d, State: Running, Role: LEADER
I20260810 06:39:58.159034 21596 sys_catalog.cc:564] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:58.159094 21601 consensus_queue.cc:237] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487477607344f44a652cad9345f2c7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260810 06:39:58.159600 21602 sys_catalog.cc:455] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b487477607344f44a652cad9345f2c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487477607344f44a652cad9345f2c7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } } }
I20260810 06:39:58.159693 21602 sys_catalog.cc:458] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:58.159955 21603 sys_catalog.cc:455] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b487477607344f44a652cad9345f2c7d. Latest consensus state: current_term: 1 leader_uuid: "b487477607344f44a652cad9345f2c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487477607344f44a652cad9345f2c7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } } }
I20260810 06:39:58.160111 21603 sys_catalog.cc:458] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:58.160334 21608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:58.160936 21608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:58.160948 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:58.149162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:47197 (local address 127.0.0.1:46797)
0810 06:39:58.149463 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:58.149472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:58.149499 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:39:58.149935 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:58.149942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:58.149989 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:58.150108 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:58.150947 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:58.151944 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:58.153012 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:58.153464 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:58.153576 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:58.156501 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:58.156505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:58.156506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:58.156519 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:58.158547 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:58.159124 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:58.159128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:58.159128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:58.159247 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:58.159539 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:58.159540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:58.159541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:58.159927 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:58.160106 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:58.160709 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 06:39:58.160798 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":171,"threads_started":1}
I20260810 06:39:58.162528 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:58.162868 21608 catalog_manager.cc:1347] Generated new cluster ID: 7afd87878d9748b6b1c826bbad0b3826
I20260810 06:39:58.162930 21608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:58.174435 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:58.167587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:60917 (local address 127.0.0.1:46797)
0810 06:39:58.167634 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:58.167637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:58.167800 (+ 163us) server_negotiation.cc:407] Connection header received
0810 06:39:58.167875 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:58.167876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:58.167887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:58.167939 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:58.168540 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:58.169060 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:58.169833 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:58.170082 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:58.170178 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:58.171594 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:58.171595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:58.171595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:39:58.171604 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:58.173143 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:58.173536 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:58.173537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:58.173538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:58.173598 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:39:58.173865 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:39:58.173867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:39:58.173867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:39:58.173986 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:58.174102 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:58.174248 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:39:58.174325 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:39:58.194568 21608 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:58.195447 21608 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:58.206840 21608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b487477607344f44a652cad9345f2c7d: Generated new TSK 0
I20260810 06:39:58.207091 21608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:58.213073 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21519
W20260810 06:39:58.222035 21501 connection.cc:535] client connection to 127.0.0.1:46797 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46797): Connection reset by peer (error 104)
2026-08-10T06:39:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:58 dist-test-slave-w206 krb5kdc[21629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:58 dist-test-slave-w206 krb5kdc[21629](info): set up 2 sockets
Aug 10 06:39:58 dist-test-slave-w206 krb5kdc[21629](info): commencing operation
krb5kdc: starting...
W20260810 06:39:59.339958 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:39:59 dist-test-slave-w206 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:39:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:39:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:39:59.473923 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:39:59.557955 21643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:39:59.558228 21643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:39:59.558270 21643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:39:59.559473 21643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:39:59.559547 21643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:39:59.559571 21643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:39:59.559593 21643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:39:59.559623 21643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:39:59.561169 21643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39537
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:39:59.561462 21643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:39:59.561725 21643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:39:59.566885 21643 server_base.cc:1034] running on GCE node
W20260810 06:39:59.566849 21649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:59.566847 21651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:39:59.566860 21648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:39:59.567436 21643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:39:59.567694 21643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:39:59.568862 21643 hybrid_clock.cc:648] HybridClock initialized: now 1786343999568848 us; error 44 us; skew 500 ppm
Aug 10 06:39:59 dist-test-slave-w206 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786343999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:39:59.571573 21643 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:39:59.572126 21643 webserver.cc:460] Webserver started at http://127.0.0.1:40359/ using document root <none> and password file <none>
I20260810 06:39:59.572391 21643 fs_manager.cc:359] Metadata directory not provided
I20260810 06:39:59.572439 21643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:39:59.572574 21643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:39:59.573647 21643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "794142c6a8cb46c295fd7f995a1ac027"
format_stamp: "Formatted at 2026-08-10 06:39:59 on dist-test-slave-w206"
server_key: "22d237aff64c13524584e47deb203ad5"
server_key_iv: "d9155673b3a1712c108f58bfb719dd5b"
server_key_version: "encryptionkey@0"
I20260810 06:39:59.574028 21643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "794142c6a8cb46c295fd7f995a1ac027"
format_stamp: "Formatted at 2026-08-10 06:39:59 on dist-test-slave-w206"
server_key: "22d237aff64c13524584e47deb203ad5"
server_key_iv: "d9155673b3a1712c108f58bfb719dd5b"
server_key_version: "encryptionkey@0"
I20260810 06:39:59.575506 21643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:39:59.576370 21659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:39:59.576560 21643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:39:59.576674 21643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "794142c6a8cb46c295fd7f995a1ac027"
format_stamp: "Formatted at 2026-08-10 06:39:59 on dist-test-slave-w206"
server_key: "22d237aff64c13524584e47deb203ad5"
server_key_iv: "d9155673b3a1712c108f58bfb719dd5b"
server_key_version: "encryptionkey@0"
I20260810 06:39:59.576756 21643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:39:59.592968 21643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:39:59.595203 21643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:39:59.595377 21643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:39:59.599838 21643 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39537
I20260810 06:39:59.599884 21716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39537 every 8 connection(s)
I20260810 06:39:59.600370 21643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:39:59.601284 21717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:39:59.603628 21717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027: Bootstrap starting.
I20260810 06:39:59.604193 21717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027: Neither blocks nor log segments found. Creating new log.
I20260810 06:39:59.604491 21717 log.cc:824] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027: Log is configured to *not* fsync() on all Append() calls
I20260810 06:39:59.605487 21717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027: No bootstrap required, opened a new log
I20260810 06:39:59.607249 21717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794142c6a8cb46c295fd7f995a1ac027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260810 06:39:59.607393 21717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:39:59.607415 21717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 794142c6a8cb46c295fd7f995a1ac027, State: Initialized, Role: FOLLOWER
I20260810 06:39:59.607517 21717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794142c6a8cb46c295fd7f995a1ac027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260810 06:39:59.607592 21717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:39:59.607628 21717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:39:59.607700 21717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:39:59.608232 21717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794142c6a8cb46c295fd7f995a1ac027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260810 06:39:59.608359 21717 leader_election.cc:302] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 794142c6a8cb46c295fd7f995a1ac027; no voters:
I20260810 06:39:59.608529 21717 leader_election.cc:288] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:39:59.608558 21720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:39:59.608749 21720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [term 1 LEADER]: Becoming Leader. State: Replica: 794142c6a8cb46c295fd7f995a1ac027, State: Running, Role: LEADER
I20260810 06:39:59.608790 21717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:39:59.608908 21720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794142c6a8cb46c295fd7f995a1ac027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260810 06:39:59.609403 21722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 794142c6a8cb46c295fd7f995a1ac027. Latest consensus state: current_term: 1 leader_uuid: "794142c6a8cb46c295fd7f995a1ac027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794142c6a8cb46c295fd7f995a1ac027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } } }
I20260810 06:39:59.609489 21722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:59.609540 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "794142c6a8cb46c295fd7f995a1ac027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794142c6a8cb46c295fd7f995a1ac027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } } }
I20260810 06:39:59.609653 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027 [sys.catalog]: This master's current role is: LEADER
I20260810 06:39:59.610035 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21643
I20260810 06:39:59.610176 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:39:59.610388 31004 external_mini_cluster.cc:1428] Setting key 08f81d85dc6639786faece57c10a10ff
W20260810 06:39:59.610718 21739 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:39:59.610834 21739 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:39:59.610946 21731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:39:59.611132 21731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:39:59.612632 21731 catalog_manager.cc:1347] Generated new cluster ID: 1f622f9cc0ab4f8686050b2fcd4c2869
I20260810 06:39:59.612689 21731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:39:59.615875 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:59.610911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:56229 (local address 127.0.0.1:39537)
0810 06:39:59.611110 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:59.611121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:59.611209 (+ 88us) server_negotiation.cc:407] Connection header received
0810 06:39:59.611383 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:59.611389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:59.611435 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:59.611571 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:59.611772 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:59.613013 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:59.613820 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:59.614276 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:59.614405 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:59.615209 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:59.615212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:59.615213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:59.615399 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:59.615429 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:59.615439 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:59.615530 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:59.615607 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 06:39:59.615653 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260810 06:39:59.617035 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:39:59.622160 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:39:59.618447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:40769 (local address 127.0.0.1:39537)
0810 06:39:59.618485 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 06:39:59.618487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:39:59.619254 (+ 767us) server_negotiation.cc:407] Connection header received
0810 06:39:59.619341 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:39:59.619342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:39:59.619353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:39:59.619400 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:39:59.619540 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:59.620052 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:59.620826 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:39:59.621036 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:39:59.621116 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:39:59.621873 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:39:59.621875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:39:59.621875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:39:59.621881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:39:59.621897 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:39:59.621902 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:39:59.621977 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:39:59.622044 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 06:39:59.622056 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:39:59.629072 21731 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:39:59.629807 21731 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:39:59.645159 21731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 794142c6a8cb46c295fd7f995a1ac027: Generated new TSK 0
I20260810 06:39:59.645411 21731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:39:59.655869 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21643
W20260810 06:39:59.661895 21627 connection.cc:535] client connection to 127.0.0.1:39537 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39537): Connection reset by peer (error 104)
2026-08-10T06:39:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:39:59 dist-test-slave-w206 krb5kdc[21751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:39:59 dist-test-slave-w206 krb5kdc[21751](info): set up 2 sockets
Aug 10 06:39:59 dist-test-slave-w206 krb5kdc[21751](info): commencing operation
krb5kdc: starting...
W20260810 06:40:00.768337 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:00 dist-test-slave-w206 krb5kdc[21751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:40:00.903677 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:00.989775 21764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:00.990061 21764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:00.990108 21764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:00.991374 21764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:00.991461 21764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:00.991493 21764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:00.991530 21764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:00.991559 21764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:00.993224 21764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:00.993548 21764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:00.993810 21764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:00.999395 21770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:00.999356 21775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:00.999396 21773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:00.999436 21764 server_base.cc:1034] running on GCE node
I20260810 06:40:00.999905 21764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:01.000252 21764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:01.001434 21764 hybrid_clock.cc:648] HybridClock initialized: now 1786344001001415 us; error 43 us; skew 500 ppm
Aug 10 06:40:01 dist-test-slave-w206 krb5kdc[21751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:01.004272 21764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:40:01.004792 21764 webserver.cc:460] Webserver started at http://127.0.0.1:41371/ using document root <none> and password file <none>
I20260810 06:40:01.005057 21764 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:01.005129 21764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:01.005283 21764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:01.006456 21764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "b5651571f8cb49fca7e0ec1f71f3d82e"
format_stamp: "Formatted at 2026-08-10 06:40:01 on dist-test-slave-w206"
server_key: "75813211c67d24b3dce9734ac2050d20"
server_key_iv: "c3d8d4bb69afeaddcafcf213f07e7c99"
server_key_version: "encryptionkey@0"
I20260810 06:40:01.006834 21764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "b5651571f8cb49fca7e0ec1f71f3d82e"
format_stamp: "Formatted at 2026-08-10 06:40:01 on dist-test-slave-w206"
server_key: "75813211c67d24b3dce9734ac2050d20"
server_key_iv: "c3d8d4bb69afeaddcafcf213f07e7c99"
server_key_version: "encryptionkey@0"
I20260810 06:40:01.008383 21764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:40:01.009213 21782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:01.009464 21764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:40:01.009579 21764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "b5651571f8cb49fca7e0ec1f71f3d82e"
format_stamp: "Formatted at 2026-08-10 06:40:01 on dist-test-slave-w206"
server_key: "75813211c67d24b3dce9734ac2050d20"
server_key_iv: "c3d8d4bb69afeaddcafcf213f07e7c99"
server_key_version: "encryptionkey@0"
I20260810 06:40:01.009658 21764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:01.013154 21764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:01.015604 21764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:01.015762 21764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:01.020061 21764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44201
I20260810 06:40:01.020103 21836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44201 every 8 connection(s)
I20260810 06:40:01.020485 21764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:01.021543 21837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:01.023957 21837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e: Bootstrap starting.
I20260810 06:40:01.024550 21837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:01.024852 21837 log.cc:824] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:01.025569 21837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e: No bootstrap required, opened a new log
I20260810 06:40:01.027290 21837 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260810 06:40:01.027431 21837 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:01.027453 21837 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5651571f8cb49fca7e0ec1f71f3d82e, State: Initialized, Role: FOLLOWER
I20260810 06:40:01.027617 21837 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260810 06:40:01.027678 21837 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:01.027699 21837 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:01.027771 21837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:01.028345 21837 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260810 06:40:01.028460 21837 leader_election.cc:302] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5651571f8cb49fca7e0ec1f71f3d82e; no voters:
I20260810 06:40:01.028649 21837 leader_election.cc:288] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:01.028692 21840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:01.028903 21840 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [term 1 LEADER]: Becoming Leader. State: Replica: b5651571f8cb49fca7e0ec1f71f3d82e, State: Running, Role: LEADER
I20260810 06:40:01.028936 21837 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:01.028996 21840 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260810 06:40:01.029392 21842 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5651571f8cb49fca7e0ec1f71f3d82e. Latest consensus state: current_term: 1 leader_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } } }
I20260810 06:40:01.029388 21841 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5651571f8cb49fca7e0ec1f71f3d82e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } } }
I20260810 06:40:01.029526 21842 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:01.029593 21841 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:01.029820 21844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:01.030084 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21764
I20260810 06:40:01.030231 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:01.030424 21844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:01.030445 31004 external_mini_cluster.cc:1428] Setting key 5fab183bec570e99f6c35960e82f270a
I20260810 06:40:01.031941 21844 catalog_manager.cc:1347] Generated new cluster ID: 18253e045c594a86b8848a5484018ab0
I20260810 06:40:01.031996 21844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:01.036370 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:01.031177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:52531 (local address 127.0.0.1:44201)
0810 06:40:01.031352 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:01.031361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:01.034678 (+ 3317us) server_negotiation.cc:407] Connection header received
0810 06:40:01.034809 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:01.034816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:01.034858 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:01.035031 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:01.035840 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:01.035844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:01.035844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:01.035847 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:01.035887 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:01.035900 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:01.035963 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:01.036048 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 06:40:01.036182 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260810 06:40:01.037356 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:01.040197 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:01.038931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:58391 (local address 127.0.0.1:44201)
0810 06:40:01.039001 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:01.039003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:01.039155 (+ 152us) server_negotiation.cc:407] Connection header received
0810 06:40:01.039189 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:01.039190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:01.039197 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:01.039271 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:01.039872 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:01.039873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:01.039873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:01.039877 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:01.039906 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:01.039910 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:01.039976 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:01.040097 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:40:01.040112 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:40:01.040359 21844 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:01.040537 21844 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:01.046093 21844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5651571f8cb49fca7e0ec1f71f3d82e: Generated new TSK 0
I20260810 06:40:01.046350 21844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:01.047989 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21764
2026-08-10T06:40:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:01 dist-test-slave-w206 krb5kdc[21872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:01 dist-test-slave-w206 krb5kdc[21872](info): set up 2 sockets
Aug 10 06:40:01 dist-test-slave-w206 krb5kdc[21872](info): commencing operation
krb5kdc: starting...
W20260810 06:40:02.149787 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:02 dist-test-slave-w206 krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:02.299530 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36591
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:36591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:02.387166 21886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:02.387422 21886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:02.387450 21886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:02.388856 21886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:02.388914 21886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:02.388931 21886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:02.388945 21886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:02.388959 21886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:02.390698 21886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:36591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:02.391029 21886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:02.391331 21886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:02.396996 21893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:02.397034 21891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:02.397034 21895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:02.397078 21886 server_base.cc:1034] running on GCE node
I20260810 06:40:02.397536 21886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:02.397833 21886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:02.399061 21886 hybrid_clock.cc:648] HybridClock initialized: now 1786344002399037 us; error 46 us; skew 500 ppm
Aug 10 06:40:02 dist-test-slave-w206 krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344002, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:02.401867 21886 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:02.402315 21886 webserver.cc:460] Webserver started at http://127.30.71.62:35721/ using document root <none> and password file <none>
I20260810 06:40:02.402576 21886 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:02.402650 21886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:02.402809 21886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:02.404050 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "998d94c7d15e4e40876ade10c71ae0a3"
format_stamp: "Formatted at 2026-08-10 06:40:02 on dist-test-slave-w206"
server_key: "32dde3b2cbee39ac4b1acc90458f25a8"
server_key_iv: "05faca8f694689b1e7d4d5c0453afb7d"
server_key_version: "encryptionkey@0"
I20260810 06:40:02.404470 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "998d94c7d15e4e40876ade10c71ae0a3"
format_stamp: "Formatted at 2026-08-10 06:40:02 on dist-test-slave-w206"
server_key: "32dde3b2cbee39ac4b1acc90458f25a8"
server_key_iv: "05faca8f694689b1e7d4d5c0453afb7d"
server_key_version: "encryptionkey@0"
I20260810 06:40:02.406075 21886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:40:02.406953 21902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:02.407203 21886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:02.407384 21886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "998d94c7d15e4e40876ade10c71ae0a3"
format_stamp: "Formatted at 2026-08-10 06:40:02 on dist-test-slave-w206"
server_key: "32dde3b2cbee39ac4b1acc90458f25a8"
server_key_iv: "05faca8f694689b1e7d4d5c0453afb7d"
server_key_version: "encryptionkey@0"
I20260810 06:40:02.407470 21886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:02.424691 21886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:02.427516 21886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:02.427722 21886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:02.432169 21886 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:36591
I20260810 06:40:02.432209 21959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:36591 every 8 connection(s)
I20260810 06:40:02.432632 21886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:02.433598 21961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:02.435902 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 21886
I20260810 06:40:02.436050 21961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3: Bootstrap starting.
I20260810 06:40:02.436081 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:02.436301 31004 external_mini_cluster.cc:1428] Setting key 18f7c998e1c413866130e6ba6fa50f82
I20260810 06:40:02.436782 21961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:02.437150 21961 log.cc:824] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:02.437891 21961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3: No bootstrap required, opened a new log
I20260810 06:40:02.440421 21961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d94c7d15e4e40876ade10c71ae0a3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36591 } }
I20260810 06:40:02.440606 21961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:02.440639 21961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998d94c7d15e4e40876ade10c71ae0a3, State: Initialized, Role: FOLLOWER
I20260810 06:40:02.440769 21961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d94c7d15e4e40876ade10c71ae0a3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36591 } }
I20260810 06:40:02.440856 21961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:02.440888 21961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:02.440949 21961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:02.441605 21961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d94c7d15e4e40876ade10c71ae0a3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36591 } }
I20260810 06:40:02.441731 21961 leader_election.cc:302] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998d94c7d15e4e40876ade10c71ae0a3; no voters:
I20260810 06:40:02.441939 21961 leader_election.cc:288] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:02.442006 21967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:02.442210 21967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [term 1 LEADER]: Becoming Leader. State: Replica: 998d94c7d15e4e40876ade10c71ae0a3, State: Running, Role: LEADER
I20260810 06:40:02.442266 21961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:02.442323 21967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d94c7d15e4e40876ade10c71ae0a3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36591 } }
I20260810 06:40:02.442689 21968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "998d94c7d15e4e40876ade10c71ae0a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d94c7d15e4e40876ade10c71ae0a3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36591 } } }
I20260810 06:40:02.442677 21967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998d94c7d15e4e40876ade10c71ae0a3. Latest consensus state: current_term: 1 leader_uuid: "998d94c7d15e4e40876ade10c71ae0a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d94c7d15e4e40876ade10c71ae0a3" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36591 } } }
I20260810 06:40:02.442770 21968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:02.442772 21967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:02.443135 21974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:02.443696 21974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:40:02 dist-test-slave-w206 krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:40:02.446180 21974 catalog_manager.cc:1347] Generated new cluster ID: 7e011d596c404a4396b853bccf18a01e
I20260810 06:40:02.446237 21974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:02.449476 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:02.436911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44077 (local address 127.30.71.62:36591)
0810 06:40:02.437127 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:02.437136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:02.437165 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:40:02.437750 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:02.437758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:02.437814 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:02.437975 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:02.438815 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:02.439862 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:02.440969 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:02.441491 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:02.441588 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:02.444999 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:02.445003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:02.445004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:02.445020 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:02.447489 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:02.448016 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:02.448019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:02.448020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:02.448135 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:02.448401 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:02.448401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:02.448402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:02.448725 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:02.448901 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:02.449099 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 06:40:02.449220 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260810 06:40:02.450954 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:02.461035 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:02.452616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58407 (local address 127.30.71.62:36591)
0810 06:40:02.452665 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:02.452667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:02.452821 (+ 154us) server_negotiation.cc:407] Connection header received
0810 06:40:02.452921 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:02.452922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:02.452931 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:02.452960 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:02.453683 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:02.454509 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:02.455636 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:02.456001 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:02.456041 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:02.457754 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:02.457756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:02.457757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:02.457775 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:02.459580 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:02.460085 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:02.460087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:02.460087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:02.460147 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:02.460388 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:02.460389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:02.460389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:02.460524 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:02.460603 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:02.460854 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 06:40:02.460926 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:40:02.464955 21974 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:02.465523 21974 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:02.468786 21974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 998d94c7d15e4e40876ade10c71ae0a3: Generated new TSK 0
I20260810 06:40:02.469031 21974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:02.472121 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 21886
W20260810 06:40:02.479431 21870 connection.cc:535] client connection to 127.30.71.62:36591 recv error: Network error: failed to read from TLS socket (remote: 127.30.71.62:36591): Connection reset by peer (error 104)
2026-08-10T06:40:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:02 dist-test-slave-w206 krb5kdc[21995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:02 dist-test-slave-w206 krb5kdc[21995](info): set up 2 sockets
Aug 10 06:40:02 dist-test-slave-w206 krb5kdc[21995](info): commencing operation
krb5kdc: starting...
W20260810 06:40:03.619307 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:03 dist-test-slave-w206 krb5kdc[21995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:03.754230 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41865
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:38565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:41865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:03.840710 22012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:03.840955 22012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:03.840982 22012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:03.842348 22012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:03.842404 22012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:03.842420 22012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:03.842433 22012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:03.842446 22012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:03.844151 22012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:38565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:41865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:03.844449 22012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:03.844710 22012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:03.849922 22018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:03.849951 22017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:03.849975 22020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:03.850205 22012 server_base.cc:1034] running on GCE node
I20260810 06:40:03.850507 22012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:03.850840 22012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:03.852038 22012 hybrid_clock.cc:648] HybridClock initialized: now 1786344003852025 us; error 45 us; skew 500 ppm
Aug 10 06:40:03 dist-test-slave-w206 krb5kdc[21995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344003, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:03.860570 22012 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:03.861129 22012 webserver.cc:460] Webserver started at http://127.30.71.62:44659/ using document root <none> and password file <none>
I20260810 06:40:03.861413 22012 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:03.861469 22012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:03.861577 22012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:03.862769 22012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "81dda4ee08cd45c1ab50799fb236eb48"
format_stamp: "Formatted at 2026-08-10 06:40:03 on dist-test-slave-w206"
server_key: "bd898630e89b2743814d473b10efca8a"
server_key_iv: "c49720bd5bbd5dd7a72e434e0ba086a6"
server_key_version: "encryptionkey@0"
I20260810 06:40:03.863219 22012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "81dda4ee08cd45c1ab50799fb236eb48"
format_stamp: "Formatted at 2026-08-10 06:40:03 on dist-test-slave-w206"
server_key: "bd898630e89b2743814d473b10efca8a"
server_key_iv: "c49720bd5bbd5dd7a72e434e0ba086a6"
server_key_version: "encryptionkey@0"
I20260810 06:40:03.864737 22012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:40:03.865537 22028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:03.865747 22012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:40:03.865846 22012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "81dda4ee08cd45c1ab50799fb236eb48"
format_stamp: "Formatted at 2026-08-10 06:40:03 on dist-test-slave-w206"
server_key: "bd898630e89b2743814d473b10efca8a"
server_key_iv: "c49720bd5bbd5dd7a72e434e0ba086a6"
server_key_version: "encryptionkey@0"
I20260810 06:40:03.865907 22012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:03.875756 22012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:03.878073 22012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:03.878203 22012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:03.882494 22012 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:41865
I20260810 06:40:03.882531 22083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:41865 every 8 connection(s)
I20260810 06:40:03.882925 22012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:03.883988 22084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:03.886543 22084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48: Bootstrap starting.
I20260810 06:40:03.887295 22084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:03.887619 22084 log.cc:824] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:03.888444 22084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48: No bootstrap required, opened a new log
I20260810 06:40:03.890353 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22012
I20260810 06:40:03.890506 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:03.890480 22084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81dda4ee08cd45c1ab50799fb236eb48" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41865 } }
I20260810 06:40:03.890702 22084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:03.890770 31004 external_mini_cluster.cc:1428] Setting key 97a3ac1ac2b10d69ab676d113ac5e0a0
I20260810 06:40:03.890765 22084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81dda4ee08cd45c1ab50799fb236eb48, State: Initialized, Role: FOLLOWER
I20260810 06:40:03.890951 22084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81dda4ee08cd45c1ab50799fb236eb48" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41865 } }
I20260810 06:40:03.891135 22084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:03.891188 22084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:03.891271 22084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:03.892351 22084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81dda4ee08cd45c1ab50799fb236eb48" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41865 } }
I20260810 06:40:03.892488 22084 leader_election.cc:302] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81dda4ee08cd45c1ab50799fb236eb48; no voters:
I20260810 06:40:03.892680 22084 leader_election.cc:288] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:03.892717 22089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:03.892844 22089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [term 1 LEADER]: Becoming Leader. State: Replica: 81dda4ee08cd45c1ab50799fb236eb48, State: Running, Role: LEADER
I20260810 06:40:03.892995 22084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:03.892995 22089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81dda4ee08cd45c1ab50799fb236eb48" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41865 } }
I20260810 06:40:03.893513 22090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81dda4ee08cd45c1ab50799fb236eb48. Latest consensus state: current_term: 1 leader_uuid: "81dda4ee08cd45c1ab50799fb236eb48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81dda4ee08cd45c1ab50799fb236eb48" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41865 } } }
I20260810 06:40:03.893616 22090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:03.893853 22091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81dda4ee08cd45c1ab50799fb236eb48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81dda4ee08cd45c1ab50799fb236eb48" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41865 } } }
I20260810 06:40:03.893990 22091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48 [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:03.894460 22096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:03.895143 22096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:03.896872 22096 catalog_manager.cc:1347] Generated new cluster ID: 1a6af46e9370447a96767d4c1f557fc9
I20260810 06:40:03.896930 22096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:03.897982 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:03.891424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50205 (local address 127.30.71.62:41865)
0810 06:40:03.891667 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:03.891678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:03.891725 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:40:03.891865 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:03.891871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:03.891917 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:03.892066 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:03.892293 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:03.893641 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:03.894947 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:03.895574 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:03.895701 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:03.896952 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:03.896957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:03.896957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:03.897286 (+ 329us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:03.897327 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:03.897344 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:03.897664 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:03.897709 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 06:40:03.897739 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260810 06:40:03.900285 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:03.906052 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:03.902037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53575 (local address 127.30.71.62:41865)
0810 06:40:03.902085 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:03.902087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:03.902168 (+ 81us) server_negotiation.cc:407] Connection header received
0810 06:40:03.902240 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:03.902241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:03.902264 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:03.902341 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:03.902510 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:03.903314 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:03.904478 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:03.904820 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:03.904906 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:03.905722 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:03.905723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:03.905724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:03.905731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:03.905751 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:03.905756 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:03.905846 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:03.905939 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 06:40:03.905954 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:40:03.948439 22096 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:03.949069 22096 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:03.956702 22096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81dda4ee08cd45c1ab50799fb236eb48: Generated new TSK 0
I20260810 06:40:03.956959 22096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:03.963192 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22012
W20260810 06:40:03.971326 21993 connection.cc:535] client connection to 127.30.71.62:41865 recv error: Network error: failed to read from TLS socket (remote: 127.30.71.62:41865): Connection reset by peer (error 104)
2026-08-10T06:40:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:04 dist-test-slave-w206 krb5kdc[22118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:04 dist-test-slave-w206 krb5kdc[22118](info): set up 2 sockets
Aug 10 06:40:04 dist-test-slave-w206 krb5kdc[22118](info): commencing operation
krb5kdc: starting...
W20260810 06:40:05.055219 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:05 dist-test-slave-w206 krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:05.191045 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33881
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:45047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:33881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:05.276283 22131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:05.276614 22131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:05.276660 22131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:05.277937 22131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:05.278029 22131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:05.278055 22131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:05.278076 22131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:05.278108 22131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:05.279775 22131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:45047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:33881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:33881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:05.280123 22131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:05.280381 22131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:05.285604 22139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:05.285632 22131 server_base.cc:1034] running on GCE node
W20260810 06:40:05.285631 22136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:05.285604 22137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:05.286092 22131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:05.286463 22131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:05.287806 22131 hybrid_clock.cc:648] HybridClock initialized: now 1786344005287760 us; error 58 us; skew 500 ppm
Aug 10 06:40:05 dist-test-slave-w206 krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:05.290779 22131 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:05.291378 22131 webserver.cc:460] Webserver started at http://127.30.71.62:35989/ using document root <none> and password file <none>
I20260810 06:40:05.291632 22131 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:05.291705 22131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:05.291872 22131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:05.293033 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "0121e43a6e69437db835b2928c76a24b"
format_stamp: "Formatted at 2026-08-10 06:40:05 on dist-test-slave-w206"
server_key: "306245e10ac1726291f14fc2bbe5b52c"
server_key_iv: "b5ee708906737428336a22b9051988bd"
server_key_version: "encryptionkey@0"
I20260810 06:40:05.293432 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "0121e43a6e69437db835b2928c76a24b"
format_stamp: "Formatted at 2026-08-10 06:40:05 on dist-test-slave-w206"
server_key: "306245e10ac1726291f14fc2bbe5b52c"
server_key_iv: "b5ee708906737428336a22b9051988bd"
server_key_version: "encryptionkey@0"
I20260810 06:40:05.294891 22131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:40:05.295790 22146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:05.295984 22131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:05.296094 22131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "0121e43a6e69437db835b2928c76a24b"
format_stamp: "Formatted at 2026-08-10 06:40:05 on dist-test-slave-w206"
server_key: "306245e10ac1726291f14fc2bbe5b52c"
server_key_iv: "b5ee708906737428336a22b9051988bd"
server_key_version: "encryptionkey@0"
I20260810 06:40:05.296176 22131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:05.299785 22131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:05.302520 22131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:05.302711 22131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:05.307466 22131 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:33881
I20260810 06:40:05.307498 22203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:33881 every 8 connection(s)
I20260810 06:40:05.307878 22131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:05.308950 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:05.311504 22204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b: Bootstrap starting.
I20260810 06:40:05.312144 22204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:05.312444 22204 log.cc:824] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:05.313208 22204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b: No bootstrap required, opened a new log
I20260810 06:40:05.314834 22204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0121e43a6e69437db835b2928c76a24b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33881 } }
I20260810 06:40:05.315074 22204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:05.315105 22204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0121e43a6e69437db835b2928c76a24b, State: Initialized, Role: FOLLOWER
I20260810 06:40:05.315268 22204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0121e43a6e69437db835b2928c76a24b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33881 } }
I20260810 06:40:05.315333 22204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:05.315356 22204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:05.315423 22204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:05.316342 22204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0121e43a6e69437db835b2928c76a24b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33881 } }
I20260810 06:40:05.316461 22204 leader_election.cc:302] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0121e43a6e69437db835b2928c76a24b; no voters:
I20260810 06:40:05.316707 22204 leader_election.cc:288] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:05.316790 22208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:05.316929 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22131
I20260810 06:40:05.316998 22208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [term 1 LEADER]: Becoming Leader. State: Replica: 0121e43a6e69437db835b2928c76a24b, State: Running, Role: LEADER
I20260810 06:40:05.317044 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:05.317091 22208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0121e43a6e69437db835b2928c76a24b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33881 } }
I20260810 06:40:05.317102 22204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:05.317250 31004 external_mini_cluster.cc:1428] Setting key 1a486fcb20eb5848bbdb65e891cf9f06
I20260810 06:40:05.317559 22210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0121e43a6e69437db835b2928c76a24b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0121e43a6e69437db835b2928c76a24b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33881 } } }
I20260810 06:40:05.317673 22210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:05.317555 22211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0121e43a6e69437db835b2928c76a24b. Latest consensus state: current_term: 1 leader_uuid: "0121e43a6e69437db835b2928c76a24b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0121e43a6e69437db835b2928c76a24b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 33881 } } }
I20260810 06:40:05.317819 22211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:05.318248 22216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:05.319213 22216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:05.320063 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:05.318003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51967 (local address 127.30.71.62:33881)
0810 06:40:05.318256 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:05.318266 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:05.318302 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:40:05.318460 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:05.318469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:05.318510 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:05.318682 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:05.319515 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:05.319519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:05.319520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:05.319525 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:05.319573 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:05.319589 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:05.319663 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:05.319807 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 06:40:05.319856 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":118,"threads_started":1}
I20260810 06:40:05.321295 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:05.321321 22216 catalog_manager.cc:1347] Generated new cluster ID: b88d6dd478f74909b9dad5aa12429505
I20260810 06:40:05.321358 22216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:05.324019 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:05.322805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38749 (local address 127.30.71.62:33881)
0810 06:40:05.322843 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:05.322845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:05.323011 (+ 166us) server_negotiation.cc:407] Connection header received
0810 06:40:05.323085 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:05.323086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:05.323092 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:05.323134 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:05.323794 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:05.323795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:05.323795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:05.323799 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:05.323813 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:05.323818 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:05.323870 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:05.323928 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 06:40:05.323941 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:40:05.329401 22216 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:05.329592 22216 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:05.341233 22216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0121e43a6e69437db835b2928c76a24b: Generated new TSK 0
I20260810 06:40:05.341508 22216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:05.346298 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22131
2026-08-10T06:40:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:05 dist-test-slave-w206 krb5kdc[22241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:05 dist-test-slave-w206 krb5kdc[22241](info): set up 2 sockets
Aug 10 06:40:05 dist-test-slave-w206 krb5kdc[22241](info): commencing operation
krb5kdc: starting...
W20260810 06:40:06.452279 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:06 dist-test-slave-w206 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:06.594905 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42479
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:42479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:06.690616 22254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:06.690862 22254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:06.690891 22254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:06.692159 22254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:06.692219 22254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:06.692236 22254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:06.692251 22254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:06.692265 22254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:06.693866 22254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:42479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:42479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:06.694173 22254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:06.694432 22254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:06.700095 22260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:06.700095 22263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:06.700117 22259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:06.700416 22254 server_base.cc:1034] running on GCE node
I20260810 06:40:06.700716 22254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:06.701081 22254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:06.702234 22254 hybrid_clock.cc:648] HybridClock initialized: now 1786344006702211 us; error 42 us; skew 500 ppm
Aug 10 06:40:06 dist-test-slave-w206 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344006, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:06.705143 22254 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:06.705657 22254 webserver.cc:460] Webserver started at http://127.30.71.62:46247/ using document root <none> and password file <none>
I20260810 06:40:06.705920 22254 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:06.705969 22254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:06.706122 22254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:06.707343 22254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "78942261c8ac4957bf3f5493555f46db"
format_stamp: "Formatted at 2026-08-10 06:40:06 on dist-test-slave-w206"
server_key: "9b34c615337150ee8b2d98f195f43b81"
server_key_iv: "dbe273c71d90d84f806e3bc6a8f1eab8"
server_key_version: "encryptionkey@0"
I20260810 06:40:06.707736 22254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "78942261c8ac4957bf3f5493555f46db"
format_stamp: "Formatted at 2026-08-10 06:40:06 on dist-test-slave-w206"
server_key: "9b34c615337150ee8b2d98f195f43b81"
server_key_iv: "dbe273c71d90d84f806e3bc6a8f1eab8"
server_key_version: "encryptionkey@0"
I20260810 06:40:06.709175 22254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:40:06.710059 22271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:06.710342 22254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:06.710453 22254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "78942261c8ac4957bf3f5493555f46db"
format_stamp: "Formatted at 2026-08-10 06:40:06 on dist-test-slave-w206"
server_key: "9b34c615337150ee8b2d98f195f43b81"
server_key_iv: "dbe273c71d90d84f806e3bc6a8f1eab8"
server_key_version: "encryptionkey@0"
I20260810 06:40:06.710526 22254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:06.725113 22254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:06.727595 22254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:06.727774 22254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:06.732170 22254 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:42479
I20260810 06:40:06.732208 22329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:42479 every 8 connection(s)
I20260810 06:40:06.732606 22254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:06.732766 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22254
I20260810 06:40:06.732928 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:06.733155 31004 external_mini_cluster.cc:1428] Setting key b11eec3f195b7ac4a107b2dbbfde11ab
I20260810 06:40:06.733757 22330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:06.737139 22330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db: Bootstrap starting.
I20260810 06:40:06.737905 22330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:06.738225 22330 log.cc:824] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:06.739158 22330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db: No bootstrap required, opened a new log
Aug 10 06:40:06 dist-test-slave-w206 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:40:06.741343 22330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78942261c8ac4957bf3f5493555f46db" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42479 } }
I20260810 06:40:06.741530 22330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:06.741575 22330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78942261c8ac4957bf3f5493555f46db, State: Initialized, Role: FOLLOWER
I20260810 06:40:06.741742 22330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78942261c8ac4957bf3f5493555f46db" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42479 } }
I20260810 06:40:06.741830 22330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:06.741874 22330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:06.741922 22330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:06.742511 22330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78942261c8ac4957bf3f5493555f46db" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42479 } }
I20260810 06:40:06.742637 22330 leader_election.cc:302] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78942261c8ac4957bf3f5493555f46db; no voters:
I20260810 06:40:06.742871 22330 leader_election.cc:288] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:06.742921 22335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:06.743093 22335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [term 1 LEADER]: Becoming Leader. State: Replica: 78942261c8ac4957bf3f5493555f46db, State: Running, Role: LEADER
I20260810 06:40:06.743188 22335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78942261c8ac4957bf3f5493555f46db" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42479 } }
I20260810 06:40:06.743737 22330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:06.743813 22336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78942261c8ac4957bf3f5493555f46db. Latest consensus state: current_term: 1 leader_uuid: "78942261c8ac4957bf3f5493555f46db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78942261c8ac4957bf3f5493555f46db" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42479 } } }
I20260810 06:40:06.743918 22336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:06.743865 22337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78942261c8ac4957bf3f5493555f46db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78942261c8ac4957bf3f5493555f46db" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 42479 } } }
I20260810 06:40:06.744062 22337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:06.744828 22339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:06.745765 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:06.733743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51389 (local address 127.30.71.62:42479)
0810 06:40:06.733971 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:06.733982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:06.734015 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:40:06.734571 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:06.734580 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:06.734657 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:06.734828 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:06.735531 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:06.736916 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:06.737675 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:06.738335 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:06.738454 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:06.741061 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:06.741067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:06.741068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:06.741081 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:06.743547 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:06.744069 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:06.744073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:06.744074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:06.744190 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:06.744437 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:06.744439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:06.744439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:06.744820 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:06.745005 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:06.745142 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:40:06.745287 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
I20260810 06:40:06.745847 22339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:06.746728 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:06.747552 22339 catalog_manager.cc:1347] Generated new cluster ID: 24a7ecd9ddcd469f9b42e1c77d763631
I20260810 06:40:06.747606 22339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:06.758913 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:06.748345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58693 (local address 127.30.71.62:42479)
0810 06:40:06.751080 (+ 2735us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:06.751081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:06.751090 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:40:06.751114 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:06.751115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:06.751126 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:06.751191 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:06.752067 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:06.752819 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:06.753876 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:06.754181 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:06.754244 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:06.755956 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:06.755957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:06.755957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:06.755965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:06.757615 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:06.758076 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:06.758077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:06.758077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:06.758137 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:06.758357 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:06.758358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:06.758358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:06.758480 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:06.758567 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:06.758735 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:40:06.758816 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2709}
I20260810 06:40:06.763902 22339 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:06.764593 22339 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:06.767500 22339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78942261c8ac4957bf3f5493555f46db: Generated new TSK 0
I20260810 06:40:06.767712 22339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:06.776379 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22254
2026-08-10T06:40:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:06 dist-test-slave-w206 krb5kdc[22361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:06 dist-test-slave-w206 krb5kdc[22361](info): set up 2 sockets
Aug 10 06:40:06 dist-test-slave-w206 krb5kdc[22361](info): commencing operation
krb5kdc: starting...
W20260810 06:40:07.858657 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:07 dist-test-slave-w206 krb5kdc[22361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:07.995384 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36175
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:36175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:08.081684 22376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:08.081948 22376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:08.081988 22376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:08.083262 22376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:08.083328 22376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:08.083353 22376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:08.083386 22376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:08.083412 22376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:08.084959 22376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:36175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:08.085260 22376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:08.085512 22376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:08.090658 22383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:08.090682 22382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:08.090668 22386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:08.090911 22376 server_base.cc:1034] running on GCE node
I20260810 06:40:08.091274 22376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:08.091614 22376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:08.092816 22376 hybrid_clock.cc:648] HybridClock initialized: now 1786344008092785 us; error 41 us; skew 500 ppm
Aug 10 06:40:08 dist-test-slave-w206 krb5kdc[22361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:08.096012 22376 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:08.096588 22376 webserver.cc:460] Webserver started at http://127.30.71.62:46103/ using document root <none> and password file <none>
I20260810 06:40:08.096881 22376 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:08.096962 22376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:08.097133 22376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:08.098407 22376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "8e9992a413854346817c27aaa4f813ad"
format_stamp: "Formatted at 2026-08-10 06:40:08 on dist-test-slave-w206"
server_key: "3972f6a7285bbc5f58ce6de737a8c006"
server_key_iv: "6b57ceb134e930c0de03a55e21422441"
server_key_version: "encryptionkey@0"
I20260810 06:40:08.098811 22376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "8e9992a413854346817c27aaa4f813ad"
format_stamp: "Formatted at 2026-08-10 06:40:08 on dist-test-slave-w206"
server_key: "3972f6a7285bbc5f58ce6de737a8c006"
server_key_iv: "6b57ceb134e930c0de03a55e21422441"
server_key_version: "encryptionkey@0"
I20260810 06:40:08.100409 22376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:40:08.101352 22394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:08.101575 22376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:40:08.101667 22376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "8e9992a413854346817c27aaa4f813ad"
format_stamp: "Formatted at 2026-08-10 06:40:08 on dist-test-slave-w206"
server_key: "3972f6a7285bbc5f58ce6de737a8c006"
server_key_iv: "6b57ceb134e930c0de03a55e21422441"
server_key_version: "encryptionkey@0"
I20260810 06:40:08.101722 22376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:08.123585 22376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:08.126171 22376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:08.126307 22376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:08.130331 22376 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:36175
I20260810 06:40:08.130415 22450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:36175 every 8 connection(s)
I20260810 06:40:08.130851 22376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:08.131597 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22376
I20260810 06:40:08.131697 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:08.131949 31004 external_mini_cluster.cc:1428] Setting key 1358dc8d0271967572e447cd1d82ea2c
I20260810 06:40:08.132028 22451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:08.135607 22451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad: Bootstrap starting.
I20260810 06:40:08.136328 22451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:08.136709 22451 log.cc:824] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:08.137579 22451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad: No bootstrap required, opened a new log
I20260810 06:40:08.138108 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:08.132512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55081 (local address 127.30.71.62:36175)
0810 06:40:08.132720 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:08.132729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:08.132764 (+ 35us) server_negotiation.cc:407] Connection header received
0810 06:40:08.133323 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:08.133330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:08.133379 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:08.133527 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:08.133810 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:08.134841 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:08.135994 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:08.136510 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:08.136598 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:08.137334 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:08.137338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:08.137339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:08.137554 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:08.137593 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:08.137620 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:08.137692 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:08.137824 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:40:08.137892 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":119,"threads_started":1}
I20260810 06:40:08.139343 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:08.139874 22451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9992a413854346817c27aaa4f813ad" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36175 } }
I20260810 06:40:08.140092 22451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:08.140131 22451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9992a413854346817c27aaa4f813ad, State: Initialized, Role: FOLLOWER
I20260810 06:40:08.140270 22451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9992a413854346817c27aaa4f813ad" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36175 } }
I20260810 06:40:08.140353 22451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:08.140383 22451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:08.140434 22451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:08.141239 22451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9992a413854346817c27aaa4f813ad" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36175 } }
I20260810 06:40:08.141366 22451 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9992a413854346817c27aaa4f813ad; no voters:
I20260810 06:40:08.141549 22451 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:08.141592 22461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:08.141758 22461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [term 1 LEADER]: Becoming Leader. State: Replica: 8e9992a413854346817c27aaa4f813ad, State: Running, Role: LEADER
I20260810 06:40:08.141853 22451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:08.141855 22461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9992a413854346817c27aaa4f813ad" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36175 } }
I20260810 06:40:08.143723 22462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e9992a413854346817c27aaa4f813ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9992a413854346817c27aaa4f813ad" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36175 } } }
I20260810 06:40:08.143821 22462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:08.143944 22463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9992a413854346817c27aaa4f813ad. Latest consensus state: current_term: 1 leader_uuid: "8e9992a413854346817c27aaa4f813ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9992a413854346817c27aaa4f813ad" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36175 } } }
I20260810 06:40:08.144012 22463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad [sys.catalog]: This master's current role is: LEADER
W20260810 06:40:08.144130 22476 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:08.144163 22476 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:08.144232 22479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:08.144407 22479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:08.145402 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:08.141103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50757 (local address 127.30.71.62:36175)
0810 06:40:08.141149 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:08.141151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:08.141309 (+ 158us) server_negotiation.cc:407] Connection header received
0810 06:40:08.141400 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:08.141401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:08.141412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:08.141460 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:08.141608 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:08.142308 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:08.143871 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:08.144174 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:08.144260 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:08.144981 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:08.144983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:08.144983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:08.144988 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:08.145005 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:08.145010 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:08.145056 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:08.145302 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 06:40:08.145325 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:40:08.146013 22479 catalog_manager.cc:1347] Generated new cluster ID: 03c3c205559a436789be40dc6e5b57f0
I20260810 06:40:08.146068 22479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:08.177632 22479 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:08.178270 22479 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:08.181810 22479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9992a413854346817c27aaa4f813ad: Generated new TSK 0
I20260810 06:40:08.182080 22479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:08.185276 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22376
2026-08-10T06:40:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:08 dist-test-slave-w206 krb5kdc[22483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:08 dist-test-slave-w206 krb5kdc[22483](info): set up 2 sockets
Aug 10 06:40:08 dist-test-slave-w206 krb5kdc[22483](info): commencing operation
krb5kdc: starting...
W20260810 06:40:09.296337 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:09 dist-test-slave-w206 krb5kdc[22483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:09.431425 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41039
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:42593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:41039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:09.516261 22497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:09.516551 22497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:09.516592 22497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:09.517841 22497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:09.517931 22497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:09.517954 22497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:09.517976 22497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:09.518008 22497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:09.519657 22497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:42593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:41039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:09.519991 22497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:09.520254 22497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:09.525611 22505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:09.525650 22497 server_base.cc:1034] running on GCE node
W20260810 06:40:09.525611 22503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:09.525640 22502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:09.526243 22497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:09.526571 22497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:09.527722 22497 hybrid_clock.cc:648] HybridClock initialized: now 1786344009527703 us; error 53 us; skew 500 ppm
Aug 10 06:40:09 dist-test-slave-w206 krb5kdc[22483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:09.536501 22497 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:09.537109 22497 webserver.cc:460] Webserver started at http://127.30.71.62:39969/ using document root <none> and password file <none>
I20260810 06:40:09.537374 22497 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:09.537458 22497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:09.537632 22497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:09.538841 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "619b824ac9714d1aa48329e102adbfde"
format_stamp: "Formatted at 2026-08-10 06:40:09 on dist-test-slave-w206"
server_key: "2f3d8a102232e76889475c5ed55ef53e"
server_key_iv: "2de6064e14e2acae59072fcbf989a1be"
server_key_version: "encryptionkey@0"
I20260810 06:40:09.539294 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "619b824ac9714d1aa48329e102adbfde"
format_stamp: "Formatted at 2026-08-10 06:40:09 on dist-test-slave-w206"
server_key: "2f3d8a102232e76889475c5ed55ef53e"
server_key_iv: "2de6064e14e2acae59072fcbf989a1be"
server_key_version: "encryptionkey@0"
I20260810 06:40:09.540781 22497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:09.541617 22514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:09.541846 22497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:09.541961 22497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "619b824ac9714d1aa48329e102adbfde"
format_stamp: "Formatted at 2026-08-10 06:40:09 on dist-test-slave-w206"
server_key: "2f3d8a102232e76889475c5ed55ef53e"
server_key_iv: "2de6064e14e2acae59072fcbf989a1be"
server_key_version: "encryptionkey@0"
I20260810 06:40:09.542039 22497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:09.545656 22497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:09.548090 22497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:09.548257 22497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:09.552510 22497 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:41039
I20260810 06:40:09.552546 22570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:41039 every 8 connection(s)
I20260810 06:40:09.552898 22497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:09.553990 22571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:09.556900 22571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde: Bootstrap starting.
I20260810 06:40:09.557430 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22497
I20260810 06:40:09.557557 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:09.557711 22571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:09.557772 31004 external_mini_cluster.cc:1428] Setting key 0517a03a0818cd42a36d7674ff74df14
I20260810 06:40:09.558092 22571 log.cc:824] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:09.558938 22571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde: No bootstrap required, opened a new log
I20260810 06:40:09.560750 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:09.558436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44403 (local address 127.30.71.62:41039)
0810 06:40:09.558613 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:09.558624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:09.558786 (+ 162us) server_negotiation.cc:407] Connection header received
0810 06:40:09.559373 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:09.559381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:09.559409 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:09.559563 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:09.560349 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:09.560353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:09.560353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:09.560357 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:09.560394 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:09.560405 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:09.560500 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:09.560534 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 06:40:09.560576 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260810 06:40:09.561677 22571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619b824ac9714d1aa48329e102adbfde" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41039 } }
I20260810 06:40:09.561884 22571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:40:09.561937 22571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619b824ac9714d1aa48329e102adbfde, State: Initialized, Role: FOLLOWER
I20260810 06:40:09.562085 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:09.562131 22571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619b824ac9714d1aa48329e102adbfde" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41039 } }
I20260810 06:40:09.562243 22571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:40:09.562285 22571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:40:09.562330 22571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:09.563201 22571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619b824ac9714d1aa48329e102adbfde" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41039 } }
I20260810 06:40:09.563338 22571 leader_election.cc:302] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 619b824ac9714d1aa48329e102adbfde; no voters:
I20260810 06:40:09.563594 22571 leader_election.cc:288] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:40:09.563639 22580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:09.563815 22580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [term 1 LEADER]: Becoming Leader. State: Replica: 619b824ac9714d1aa48329e102adbfde, State: Running, Role: LEADER
I20260810 06:40:09.563884 22571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:09.563942 22580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619b824ac9714d1aa48329e102adbfde" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41039 } }
I20260810 06:40:09.564432 22580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 619b824ac9714d1aa48329e102adbfde. Latest consensus state: current_term: 1 leader_uuid: "619b824ac9714d1aa48329e102adbfde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619b824ac9714d1aa48329e102adbfde" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41039 } } }
I20260810 06:40:09.564522 22580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:09.564481 22581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "619b824ac9714d1aa48329e102adbfde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619b824ac9714d1aa48329e102adbfde" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 41039 } } }
I20260810 06:40:09.564754 22586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:09.564991 22581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:09.565404 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:09.563888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60759 (local address 127.30.71.62:41039)
0810 06:40:09.563940 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:09.563942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:09.564177 (+ 235us) server_negotiation.cc:407] Connection header received
0810 06:40:09.564288 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:09.564289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:09.564294 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:09.564368 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:09.565113 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:09.565114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:09.565115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:40:09.565119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:09.565135 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:40:09.565139 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:09.565183 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:09.565336 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 06:40:09.565348 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 06:40:09.565536 22586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:09.567214 22586 catalog_manager.cc:1347] Generated new cluster ID: 8082075986cf4e73ba3d8b5a31b83c64
I20260810 06:40:09.567296 22586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:09.597790 22586 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:09.598058 22586 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:09.623484 22586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 619b824ac9714d1aa48329e102adbfde: Generated new TSK 0
I20260810 06:40:09.623752 22586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:09.630379 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22497
2026-08-10T06:40:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1445 ms)
[----------] 18 tests from AuthTokenIssuingTest (25627 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:09 dist-test-slave-w206 krb5kdc[22602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:09 dist-test-slave-w206 krb5kdc[22602](info): set up 2 sockets
Aug 10 06:40:09 dist-test-slave-w206 krb5kdc[22602](info): commencing operation
krb5kdc: starting...
W20260810 06:40:10.718891 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:10 dist-test-slave-w206 krb5kdc[22602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:10.855499 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36279
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:36279,127.30.71.61:32861,127.30.71.60:39299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:10.941200 22615 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:40:10.941495 22615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:10.941538 22615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:10.941581 22615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:10.942812 22615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:10.942890 22615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:10.942916 22615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:10.942950 22615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:10.943003 22615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:10.944612 22615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:36279,127.30.71.61:32861,127.30.71.60:39299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:36279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:10.944929 22615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:10.945175 22615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:10.950935 22620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:10.951011 22621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:10.951023 22615 server_base.cc:1034] running on GCE node
W20260810 06:40:10.951045 22623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:10.951510 22615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:10.951839 22615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:10.953022 22615 hybrid_clock.cc:648] HybridClock initialized: now 1786344010953000 us; error 46 us; skew 500 ppm
Aug 10 06:40:10 dist-test-slave-w206 krb5kdc[22602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:10.955895 22615 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:10.956424 22615 webserver.cc:460] Webserver started at http://127.30.71.62:43075/ using document root <none> and password file <none>
I20260810 06:40:10.956707 22615 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:10.956779 22615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:10.956954 22615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:10.958149 22615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "220aab69074a4343a2e0414c4facff4b"
format_stamp: "Formatted at 2026-08-10 06:40:10 on dist-test-slave-w206"
server_key: "26b029512ef50906e5b95a8d45b2691a"
server_key_iv: "883d82a688308b8c506080c29e5d0cbd"
server_key_version: "encryptionkey@0"
I20260810 06:40:10.958556 22615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "220aab69074a4343a2e0414c4facff4b"
format_stamp: "Formatted at 2026-08-10 06:40:10 on dist-test-slave-w206"
server_key: "26b029512ef50906e5b95a8d45b2691a"
server_key_iv: "883d82a688308b8c506080c29e5d0cbd"
server_key_version: "encryptionkey@0"
I20260810 06:40:10.960121 22615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 06:40:10.961047 22631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:10.961313 22615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:10.961449 22615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "220aab69074a4343a2e0414c4facff4b"
format_stamp: "Formatted at 2026-08-10 06:40:10 on dist-test-slave-w206"
server_key: "26b029512ef50906e5b95a8d45b2691a"
server_key_iv: "883d82a688308b8c506080c29e5d0cbd"
server_key_version: "encryptionkey@0"
I20260810 06:40:10.961539 22615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:10.978278 22615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:10.980702 22615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:10.980890 22615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:10.985297 22615 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:36279
I20260810 06:40:10.985339 22687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:36279 every 8 connection(s)
I20260810 06:40:10.985714 22615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:10.986778 22689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:10.988489 22689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:10.991843 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22615
I20260810 06:40:10.991953 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:10.992249 31004 external_mini_cluster.cc:1428] Setting key 0c9a037b04df232ccf9370a76f984330
Aug 10 06:40:10 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:40:11.000212 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:10.989370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36066 (local address 127.30.71.62:36279)
0810 06:40:10.989499 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:10.989509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:10.989601 (+ 92us) server_negotiation.cc:407] Connection header received
0810 06:40:10.990266 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:10.990268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:10.990297 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:10.990411 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:10.991096 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:10.992122 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:10.993188 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:10.993545 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:10.993685 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:10.996464 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:10.996474 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:10.996474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:10.996490 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:10.998319 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:10.998782 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:10.998786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:10.998786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:10.998881 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:10.999167 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:10.999168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:10.999169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:10.999406 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:10.999512 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:10.999740 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 06:40:10.999997 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
I20260810 06:40:11.000214 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:10.989192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:36279 (local address 127.0.0.1:36066)
0810 06:40:10.989418 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:10.989498 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:10.990065 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:10.990433 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:10.990436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:10.990764 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:10.990933 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:10.990938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:10.992245 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:10.992246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:10.993080 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:10.993083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:10.993676 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:10.993678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:10.993749 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:10.994422 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:10.994432 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:10.996335 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:10.998456 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:10.998464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:10.998466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:10.998678 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:10.998961 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:10.998962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:10.998962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:10.999090 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:10.999498 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:10.999504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:10.999573 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 06:40:10.999688 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 06:40:10.999933 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.30.71.61@KRBTEST.COM; defaulting to no policy
I20260810 06:40:11.001587 22689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:11.002076 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.001949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59748)
0810 06:40:11.002003 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.002051 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:40:11.002249 22633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
I20260810 06:40:11.002790 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.002665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59764)
0810 06:40:11.002730 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.002767 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:40:11.002952 22689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:32861: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab.
Entry for principal kudu/127.30.71.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab.
I20260810 06:40:11.022758 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:32861
--webserver_interface=127.30.71.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab
--principal=kudu/127.30.71.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:36279,127.30.71.61:32861,127.30.71.60:39299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:11.052300 22689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } attempt: 1
I20260810 06:40:11.052930 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.052750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59772)
0810 06:40:11.052834 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.052883 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:40:11.053419 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.053294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59786)
0810 06:40:11.053352 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.053394 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:40:11.053527 22689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:32861: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
W20260810 06:40:11.107442 22697 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:40:11.107712 22697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:11.107743 22697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:11.107785 22697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:11.108986 22697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:11.109063 22697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:11.109087 22697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:11.109109 22697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:11.109140 22697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:11.110692 22697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:36279,127.30.71.61:32861,127.30.71.60:39299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab
--principal=kudu/127.30.71.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:32861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.71.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:11.111019 22697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:11.111281 22697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:11.116657 22705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:11.116647 22707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:11.116761 22697 server_base.cc:1034] running on GCE node
W20260810 06:40:11.116645 22704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:11.117195 22697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:11.117553 22697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:11.118762 22697 hybrid_clock.cc:648] HybridClock initialized: now 1786344011118729 us; error 54 us; skew 500 ppm
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:11.121708 22689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } attempt: 2
I20260810 06:40:11.121928 22697 init.cc:377] Logged in from keytab as kudu/127.30.71.61@KRBTEST.COM (short username kudu)
I20260810 06:40:11.122197 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.122014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59796)
0810 06:40:11.122095 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.122154 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:40:11.122571 22697 webserver.cc:460] Webserver started at http://127.30.71.61:39785/ using document root <none> and password file <none>
I20260810 06:40:11.122587 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.122474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59806)
0810 06:40:11.122530 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.122560 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:40:11.122740 22689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:32861: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32861: connect: Connection refused (error 111)
I20260810 06:40:11.122934 22697 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:11.123050 22697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:11.123281 22697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:11.124573 22697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data/instance:
uuid: "71d234694d91498e9e46099eaee5f0df"
format_stamp: "Formatted at 2026-08-10 06:40:11 on dist-test-slave-w206"
server_key: "c18b8d48adb0c0195ca4dec6fff4a0a7"
server_key_iv: "60b316a642581c39c14c9f6f11110f61"
server_key_version: "encryptionkey@0"
I20260810 06:40:11.124992 22697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/wal/instance:
uuid: "71d234694d91498e9e46099eaee5f0df"
format_stamp: "Formatted at 2026-08-10 06:40:11 on dist-test-slave-w206"
server_key: "c18b8d48adb0c0195ca4dec6fff4a0a7"
server_key_iv: "60b316a642581c39c14c9f6f11110f61"
server_key_version: "encryptionkey@0"
I20260810 06:40:11.126621 22697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:40:11.127628 22715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:11.127851 22697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 06:40:11.127938 22697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/wal
uuid: "71d234694d91498e9e46099eaee5f0df"
format_stamp: "Formatted at 2026-08-10 06:40:11 on dist-test-slave-w206"
server_key: "c18b8d48adb0c0195ca4dec6fff4a0a7"
server_key_iv: "60b316a642581c39c14c9f6f11110f61"
server_key_version: "encryptionkey@0"
I20260810 06:40:11.127995 22697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:11.151984 22697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:11.154613 22697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:11.154745 22697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:11.158865 22697 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.61:32861
I20260810 06:40:11.159112 22769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.61:32861 every 8 connection(s)
I20260810 06:40:11.159420 22697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
I20260810 06:40:11.160451 22770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:11.162364 22770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:40:11.169365 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22697
I20260810 06:40:11.169473 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-1/wal/instance
I20260810 06:40:11.169744 31004 external_mini_cluster.cc:1428] Setting key eba1a762879aea33768ef4ecd5de8a8d
I20260810 06:40:11.173059 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.163320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.30.71.62:36279)
0810 06:40:11.163413 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.163416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.163543 (+ 127us) server_negotiation.cc:407] Connection header received
0810 06:40:11.164148 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.164149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.164161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.164237 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.165027 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.165632 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.166841 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.167124 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.167185 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.169811 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.169813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.169813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.169823 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.171471 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.171986 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.171987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.171988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.172052 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.172336 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.172337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.172337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.172464 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.172546 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.172908 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.172990 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:40:11.173362 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.163164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:36279 (local address 127.0.0.1:36082)
0810 06:40:11.163395 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.163442 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.164048 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.164299 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.164303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.164659 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.164922 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.164928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.165774 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.165775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.166731 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.166734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.167237 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.167239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.167372 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.167942 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.167952 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.169699 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.171603 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.171613 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.171616 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.171889 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.172121 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.172122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.172123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.172255 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.172536 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.172543 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.172734 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.172866 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.173066 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260810 06:40:11.174110 22770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.71.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.60@KRBTEST.COM" created.
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:40:11.186465 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.174440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59808)
0810 06:40:11.174478 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.174489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.174536 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.174859 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.174861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.175082 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.175160 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.175162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.176239 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.176240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.177089 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.177093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.177713 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.177714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.177798 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.178388 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.178395 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.183132 (+ 4737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.185272 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.185273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.185273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.185478 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.185717 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.185718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.185718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.185788 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.186111 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.186112 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.186155 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.186219 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.186342 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 06:40:11.186494 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.174512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.30.71.61:32861)
0810 06:40:11.174684 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.174688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.174708 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:40:11.174728 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.174732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.174749 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.174819 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.175373 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.176133 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.177192 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.177619 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.177686 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.183264 (+ 5578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.183289 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.183290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.183297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.185143 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.185570 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.185575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.185575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.185637 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.185849 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.185850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.185851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.186029 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.186127 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.186248 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.186397 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
I20260810 06:40:11.187323 22770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:11.187726 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.187589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49788)
0810 06:40:11.187645 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.187697 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 06:40:11.187906 22718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
I20260810 06:40:11.188374 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.188278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49802)
0810 06:40:11.188327 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.188351 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:40:11.188488 22770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:39299: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab.
Entry for principal kudu/127.30.71.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab.
I20260810 06:40:11.203421 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:39299
--webserver_interface=127.30.71.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab
--principal=kudu/127.30.71.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:36279,127.30.71.61:32861,127.30.71.60:39299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:11.213941 22689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } attempt: 3
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:40:11.221508 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.214535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.30.71.61:32861)
0810 06:40:11.214607 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.214609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.214621 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:40:11.214671 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.214672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.214681 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.214748 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.215173 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.215698 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.216421 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.216644 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.216725 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.218872 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.218874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.218874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.218882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.220371 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.220794 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.220795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.220795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.220850 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.221048 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.221049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.221049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.221154 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.221204 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.221369 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.221436 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:40:11.221558 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.214420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59816)
0810 06:40:11.214500 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.214520 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.214595 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.214768 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.214769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.214948 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.215085 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.215087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.215796 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.215798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.216328 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.216330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.216723 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.216724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.216823 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.217342 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.217349 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.218754 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.220457 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.220458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.220459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.220701 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.220915 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.220916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.220916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.220987 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.221213 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.221214 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.221268 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.221340 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.221452 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:40:11.222082 22689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:11.222479 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.222362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49810)
0810 06:40:11.222413 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.222451 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 06:40:11.222839 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.222728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49814)
0810 06:40:11.222789 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.222816 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 06:40:11.222963 22689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:39299: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
I20260810 06:40:11.237730 22770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } attempt: 1
I20260810 06:40:11.238281 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.238128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49818)
0810 06:40:11.238198 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.238242 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:40:11.238675 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.238569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49828)
0810 06:40:11.238621 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.238649 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:40:11.238781 22770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:39299: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
I20260810 06:40:11.254156 22689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } attempt: 1
I20260810 06:40:11.254721 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.254549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49834)
0810 06:40:11.254626 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.254680 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:40:11.255193 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.255089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49850)
0810 06:40:11.255141 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.255171 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:40:11.255456 22689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:39299: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
W20260810 06:40:11.291623 22778 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:40:11.291862 22778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:11.291879 22778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:11.291901 22778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:11.293128 22778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:11.293184 22778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:11.293200 22778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:11.293212 22778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:11.293224 22778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:11.294858 22778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:36279,127.30.71.61:32861,127.30.71.60:39299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab
--principal=kudu/127.30.71.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:39299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.71.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:11.295166 22778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:11.295481 22778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:11.300550 22783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:11.300602 22786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:11.300616 22784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:11.301152 22778 server_base.cc:1034] running on GCE node
I20260810 06:40:11.301426 22778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:11.301756 22778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:11.302932 22778 hybrid_clock.cc:648] HybridClock initialized: now 1786344011302911 us; error 43 us; skew 500 ppm
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:11.305991 22778 init.cc:377] Logged in from keytab as kudu/127.30.71.60@KRBTEST.COM (short username kudu)
I20260810 06:40:11.306519 22778 webserver.cc:460] Webserver started at http://127.30.71.60:36253/ using document root <none> and password file <none>
I20260810 06:40:11.306787 22778 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:11.306838 22778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:11.306945 22770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } attempt: 2
I20260810 06:40:11.307066 22778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:11.307401 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.307244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49862)
0810 06:40:11.307315 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.307359 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:40:11.307797 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.307693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49870)
0810 06:40:11.307741 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.307772 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 06:40:11.307907 22770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:39299: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
I20260810 06:40:11.308490 22778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data/instance:
uuid: "780dcb8b1aa34a69801255cead4f3d45"
format_stamp: "Formatted at 2026-08-10 06:40:11 on dist-test-slave-w206"
server_key: "f93f1fcc8aecba097c1c71905fd9b6b9"
server_key_iv: "2dde2f50db421936821c7f2ff5a04009"
server_key_version: "encryptionkey@0"
I20260810 06:40:11.308908 22778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/wal/instance:
uuid: "780dcb8b1aa34a69801255cead4f3d45"
format_stamp: "Formatted at 2026-08-10 06:40:11 on dist-test-slave-w206"
server_key: "f93f1fcc8aecba097c1c71905fd9b6b9"
server_key_iv: "2dde2f50db421936821c7f2ff5a04009"
server_key_version: "encryptionkey@0"
I20260810 06:40:11.310343 22778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:40:11.311185 22795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:11.311385 22778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:11.311487 22778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/wal
uuid: "780dcb8b1aa34a69801255cead4f3d45"
format_stamp: "Formatted at 2026-08-10 06:40:11 on dist-test-slave-w206"
server_key: "f93f1fcc8aecba097c1c71905fd9b6b9"
server_key_iv: "2dde2f50db421936821c7f2ff5a04009"
server_key_version: "encryptionkey@0"
I20260810 06:40:11.311569 22778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:11.330278 22778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:11.330646 22689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } attempt: 2
I20260810 06:40:11.331166 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.331035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49874)
0810 06:40:11.331089 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.331132 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 06:40:11.331629 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.331495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49886)
0810 06:40:11.331543 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.331569 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:40:11.331749 22689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:39299: Network error: Client connection negotiation failed: client connection to 127.30.71.60:39299: connect: Connection refused (error 111)
I20260810 06:40:11.333442 22778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:11.333570 22778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:11.337890 22778 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.60:39299
I20260810 06:40:11.337966 22850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.60:39299 every 8 connection(s)
I20260810 06:40:11.338346 22778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
I20260810 06:40:11.339347 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 22778
I20260810 06:40:11.339430 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786343734291041-31004-0/minicluster-data/master-2/wal/instance
I20260810 06:40:11.339532 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:11.339663 31004 external_mini_cluster.cc:1428] Setting key d31535e6a0c6902356365bba75f39c93
I20260810 06:40:11.342125 22851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
I20260810 06:40:11.351178 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.342908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.30.71.62:36279)
0810 06:40:11.343004 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.343006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.343142 (+ 136us) server_negotiation.cc:407] Connection header received
0810 06:40:11.343234 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.343235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.343246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.343301 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.343801 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.344253 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.345075 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.345259 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.345344 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.347556 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.347558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.347558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.347566 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.349274 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.350189 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.350190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.350190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.350239 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.350505 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.350506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.350506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.350653 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.350706 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.350967 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.351082 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:40:11.351382 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.340414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49902 (local address 127.30.71.60:39299)
0810 06:40:11.340600 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.340610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.340650 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:40:11.341148 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.341155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.341215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.341343 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.342277 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.343564 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.344427 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.344882 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.345029 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.347997 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.348005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.348005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.348013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.349856 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.350183 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.350186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.350186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.350288 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.350558 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.350558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.350558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.350823 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.351017 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.351119 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.351180 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260810 06:40:11.351393 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.342834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:36279 (local address 127.0.0.1:36090)
0810 06:40:11.343035 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.343078 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.343160 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.343340 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.343343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.343644 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.343707 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.343708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.344345 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.344346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.344982 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.344985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.345368 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.345369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.345440 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.345860 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.345868 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.347437 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.349381 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.349386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.349389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.349851 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.350304 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.350305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.350305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.350419 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.350722 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.350726 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.350818 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.350919 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.351129 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":93,"threads_started":1}
I20260810 06:40:11.352414 22851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:40:11.359596 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.352800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.30.71.61:32861)
0810 06:40:11.352863 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.352865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.352873 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:40:11.352899 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.352900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.352910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.352975 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.353232 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.353790 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.354439 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.354662 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.354726 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.356642 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.356643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.356644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.356650 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.358317 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.358703 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.358704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.358705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.358760 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.358966 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.358967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.358967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.359141 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.359208 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.359407 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.359494 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:40:11.359596 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.352664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32861 (local address 127.0.0.1:59818)
0810 06:40:11.352695 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.352703 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.352729 (+ 26us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.352985 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.352986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.353100 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.353153 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.353154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.353873 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.353873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.354338 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.354340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.354744 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.354745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.354799 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.355324 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.355348 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.356557 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.358405 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.358407 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.358407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.358626 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.358828 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.358829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.358829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.358887 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.359214 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.359215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.359266 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.359373 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.359484 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 06:40:11.360061 22851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:11.360496 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.352385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.30.71.62:36279)
0810 06:40:11.352422 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.352424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.352467 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:40:11.352556 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.352558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.352570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.352630 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.353257 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.353704 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.354410 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.354610 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.354661 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.357209 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.357211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.357211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.357219 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.358612 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.359161 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.359162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.359162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.359240 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.359507 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.359508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.359509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.359643 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.359844 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.360292 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.360397 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:40:11.367934 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.360421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.30.71.60:39299)
0810 06:40:11.360457 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.360459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.360467 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:40:11.360506 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.360507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.360516 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.360568 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.360841 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.361536 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.362523 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.362849 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.362929 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.365226 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.365228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.365228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.365235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.366819 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.367246 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.367247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.367247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.367303 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.367500 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.367501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.367501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.367589 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.367633 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.367801 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.367867 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:40:11.368048 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.360356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49904)
0810 06:40:11.360406 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.360417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.360455 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.360571 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.360572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.360731 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.360765 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.360767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.361620 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.361621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.362396 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.362399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.362932 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.362932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.363010 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.363514 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.363518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.365143 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.366903 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.366904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.366905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.367170 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.367370 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.367372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.367372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.367438 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.367650 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.367651 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.367700 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.367784 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.367938 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":45}
I20260810 06:40:11.369168 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.361182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.30.71.61:32861)
0810 06:40:11.361222 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.361224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.361232 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:40:11.361299 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.361300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.361308 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.361365 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.361784 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.362451 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.363315 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.363599 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.363652 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.366202 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.366204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.366204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.366211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.367885 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.368443 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.368444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.368444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.368533 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.368742 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.368742 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.368742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.368838 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.368949 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.369052 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.369107 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 06:40:11.369990 22851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: Bootstrap starting.
I20260810 06:40:11.370865 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:11.371176 22851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:11.371488 22851 log.cc:824] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:11.372375 22851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: No bootstrap required, opened a new log
I20260810 06:40:11.371868 22662 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "220aab69074a4343a2e0414c4facff4b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36096
I20260810 06:40:11.373950 22851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.374105 22851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 780dcb8b1aa34a69801255cead4f3d45, State: Initialized, Role: FOLLOWER
I20260810 06:40:11.374199 22851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.374541 22851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:11.374511 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.374608 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:40:11.376452 22869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:11.376513 22869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:11.376756 22778 master_runner.cc:186] Error getting master registration for 127.30.71.62:36279: OK, instance_id { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" instance_seqno: 1786344010984067 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:40:11.379842 22778 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:40:11.399135 22770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } attempt: 3
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
I20260810 06:40:11.407533 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.399637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49920)
0810 06:40:11.399717 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.399742 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.399822 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.400023 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.400024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.400219 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.400276 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.400279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.401034 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.401036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.401587 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.401589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.401990 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.401991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.402066 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.402593 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.402599 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.404293 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.406184 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.406185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.406186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.406432 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.406694 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.406695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.406695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.406765 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.407179 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.407180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.407234 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.407307 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.407424 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:40:11.407539 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.399751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.30.71.60:39299)
0810 06:40:11.399800 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.399803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.399814 (+ 11us) server_negotiation.cc:407] Connection header received
0810 06:40:11.399925 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.399926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.399937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.400007 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.400382 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.400920 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.401687 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.401903 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.401993 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.404433 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.404435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.404435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.404445 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.406097 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.406529 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.406531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.406531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.406605 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.406848 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.406849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.406850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.407093 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.407166 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.407344 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.407437 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 06:40:11.409898 22770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: Bootstrap starting.
I20260810 06:40:11.410589 22770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:11.410917 22770 log.cc:824] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:11.411795 22770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: No bootstrap required, opened a new log
I20260810 06:40:11.413261 22770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.413420 22770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d234694d91498e9e46099eaee5f0df, State: Initialized, Role: FOLLOWER
I20260810 06:40:11.413532 22770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.413852 22872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.413926 22770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:11.413950 22872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:11.416247 22697 master_runner.cc:186] Error getting master registration for 127.30.71.62:36279: OK, instance_id { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" instance_seqno: 1786344010984067 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:40:11.418430 22697 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 06:40:11.419663 22886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:11.419718 22886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:11.430940 22689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } attempt: 3
Aug 10 06:40:11 dist-test-slave-w206 krb5kdc[22602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
I20260810 06:40:11.439327 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.431387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:39299 (local address 127.0.0.1:49936)
0810 06:40:11.431442 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:11.431456 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:11.431511 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:11.431725 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:11.431726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:11.431926 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:11.431973 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.431975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.432880 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.432881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.433400 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:11.433401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.433889 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.433890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:11.433953 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.434453 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:11.434461 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:11.436032 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:11.438148 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.438150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.438150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.438338 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.438564 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:11.438567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:11.438567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:11.438623 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:11.438924 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:11.438926 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:11.439020 (+ 94us) client_negotiation.cc:769] Sending connection context
0810 06:40:11.439109 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:40:11.439222 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 06:40:11.439327 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.431546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.30.71.60:39299)
0810 06:40:11.431605 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.431607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.431615 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:40:11.431644 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.431645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.431656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.431717 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.432065 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.432802 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.433504 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.433811 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.433878 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.436138 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.436139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.436140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.436148 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.438063 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.438421 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.438423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.438423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.438487 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.438705 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.438707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.438707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.438855 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.438914 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.439138 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.439222 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:40:11.441707 22689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b: Bootstrap starting.
I20260810 06:40:11.442478 22689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:11.442770 22689 log.cc:824] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:11.443629 22689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b: No bootstrap required, opened a new log
I20260810 06:40:11.444711 22689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.444859 22689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220aab69074a4343a2e0414c4facff4b, State: Initialized, Role: FOLLOWER
I20260810 06:40:11.444947 22689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.445264 22889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.445385 22662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:40:11.445333 22689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:11.445405 22889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:11.445444 22662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:11.447615 22662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.447923 22662 leader_election.cc:288] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [CANDIDATE]: Term 1 election: Requested vote from peers 71d234694d91498e9e46099eaee5f0df (127.30.71.61:32861), 780dcb8b1aa34a69801255cead4f3d45 (127.30.71.60:39299)
I20260810 06:40:11.448524 22744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "220aab69074a4343a2e0414c4facff4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "71d234694d91498e9e46099eaee5f0df"
I20260810 06:40:11.448685 22744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:11.449748 22744 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220aab69074a4343a2e0414c4facff4b in term 1.
I20260810 06:40:11.450671 22823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "220aab69074a4343a2e0414c4facff4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "780dcb8b1aa34a69801255cead4f3d45"
I20260810 06:40:11.449977 22633 leader_election.cc:302] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220aab69074a4343a2e0414c4facff4b, 71d234694d91498e9e46099eaee5f0df; no voters:
I20260810 06:40:11.450814 22823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:11.450927 22889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:11.451108 22889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [term 1 LEADER]: Becoming Leader. State: Replica: 220aab69074a4343a2e0414c4facff4b, State: Running, Role: LEADER
I20260810 06:40:11.451195 22889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } }
I20260810 06:40:11.451642 22889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220aab69074a4343a2e0414c4facff4b. Latest consensus state: current_term: 1 leader_uuid: "220aab69074a4343a2e0414c4facff4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.451711 22889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:11.451890 22823 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220aab69074a4343a2e0414c4facff4b in term 1.
I20260810 06:40:11.453861 22615 master_runner.cc:263] No leader master found from master 220aab69074a4343a2e0414c4facff4b
I20260810 06:40:11.453908 22615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 06:40:11.454849 22900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:11.454996 22900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:11.458946 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.449374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.30.71.62:36279)
0810 06:40:11.449421 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.449423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.450770 (+ 1347us) server_negotiation.cc:407] Connection header received
0810 06:40:11.450880 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.450881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.450893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.450925 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.451909 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.452638 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.453725 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.454036 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.454085 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.455887 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.455889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.455889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.455896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.457600 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.458037 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.458038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.458038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.458097 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.458316 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.458317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.458317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.458446 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.458534 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.458775 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.458855 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:40:11.459669 22910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:11.459919 22910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:11.465555 22823 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [term 1 FOLLOWER]: Refusing update from remote peer 220aab69074a4343a2e0414c4facff4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:11.465914 22889 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:11.467253 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220aab69074a4343a2e0414c4facff4b. Latest consensus state: current_term: 1 leader_uuid: "220aab69074a4343a2e0414c4facff4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.467360 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:11.467726 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.450312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.30.71.60:39299)
0810 06:40:11.450392 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.450418 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.459306 (+ 8888us) server_negotiation.cc:407] Connection header received
0810 06:40:11.459416 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.459417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.459430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.459460 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.460289 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.461030 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.462092 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.462496 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.462545 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.464344 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.464345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.464346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.464354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.466111 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.466667 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.466680 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.466681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.466742 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.466954 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.466955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.466955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.467120 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.467196 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.467535 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.467620 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:40:11.468003 22744 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [term 1 FOLLOWER]: Refusing update from remote peer 220aab69074a4343a2e0414c4facff4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:11.468350 22908 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:11.470187 22872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220aab69074a4343a2e0414c4facff4b. Latest consensus state: current_term: 1 leader_uuid: "220aab69074a4343a2e0414c4facff4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.470302 22872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:11.471112 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:11.449782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.30.71.61:32861)
0810 06:40:11.449822 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.449824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.459185 (+ 9361us) server_negotiation.cc:407] Connection header received
0810 06:40:11.460333 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.460334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.460348 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.460380 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.461245 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.461934 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.462959 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:11.463305 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:11.463347 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:11.467833 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:11.467835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:11.467835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:11.467844 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:11.469587 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.470007 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.470008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.470008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.470082 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:11.470442 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:11.470444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:11.470444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:11.470593 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:11.470665 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:11.470901 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 06:40:11.471023 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 06:40:11.481762 22638 tablet.cc:2369] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:40:11.482520 22889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "220aab69074a4343a2e0414c4facff4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.482643 22889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:11.482702 22912 mvcc.cc:205] Tried to move back new op lower bound from 7316865070960046080 to 7316865070904852480. Current Snapshot: MvccSnapshot[applied={T|T < 7316865070960046080}]
I20260810 06:40:11.482743 22915 mvcc.cc:205] Tried to move back new op lower bound from 7316865070960046080 to 7316865070904852480. Current Snapshot: MvccSnapshot[applied={T|T < 7316865070960046080}]
I20260810 06:40:11.482793 22872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "220aab69074a4343a2e0414c4facff4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.482862 22872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:11.483609 22908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "220aab69074a4343a2e0414c4facff4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.483700 22908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:11.483948 22910 catalog_manager.cc:1347] Generated new cluster ID: a926ced8addc463db10799de14a4656b
I20260810 06:40:11.484009 22910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:11.484145 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "220aab69074a4343a2e0414c4facff4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220aab69074a4343a2e0414c4facff4b" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 36279 } } peers { permanent_uuid: "71d234694d91498e9e46099eaee5f0df" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32861 } } peers { permanent_uuid: "780dcb8b1aa34a69801255cead4f3d45" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 39299 } } }
I20260810 06:40:11.484237 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:11.484560 22913 mvcc.cc:205] Tried to move back new op lower bound from 7316865070960046080 to 7316865070904852480. Current Snapshot: MvccSnapshot[applied={T|T < 7316865070960046080}]
I20260810 06:40:11.516891 22910 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:11.517706 22910 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:11.532948 22910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 220aab69074a4343a2e0414c4facff4b: Generated new TSK 0
I20260810 06:40:11.533211 22910 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:40:11.566500 22692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.564379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.30.71.62:36279)
0810 06:40:11.564434 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.564436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.565666 (+ 1230us) server_negotiation.cc:407] Connection header received
0810 06:40:11.565796 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.565797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.565812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.565849 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.566415 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36120: BlockingRecv error: recv got EOF from 127.0.0.1:36120 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 06:40:11.567447 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.564891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.30.71.61:32861)
0810 06:40:11.564938 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.564940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.566808 (+ 1868us) server_negotiation.cc:407] Connection header received
0810 06:40:11.566927 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.566928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.566941 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.567018 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.567379 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59858: BlockingRecv error: recv got EOF from 127.0.0.1:59858 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 06:40:11.568320 22855 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.565305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.30.71.60:39299)
0810 06:40:11.565349 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.565351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.567694 (+ 2343us) server_negotiation.cc:407] Connection header received
0810 06:40:11.567873 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.567875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.567887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.567929 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.568259 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49954: BlockingRecv error: recv got EOF from 127.0.0.1:49954 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 06:40:11.570930 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.570104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.30.71.61:32861)
0810 06:40:11.570146 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.570148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.570361 (+ 213us) server_negotiation.cc:407] Connection header received
0810 06:40:11.570474 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.570475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.570496 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.570529 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.570879 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59872: BlockingRecv error: recv got EOF from 127.0.0.1:59872 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 06:40:11.574234 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.573377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59884 (local address 127.30.71.61:32861)
0810 06:40:11.573417 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.573419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.573693 (+ 274us) server_negotiation.cc:407] Connection header received
0810 06:40:11.573801 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.573802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.573814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.573847 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.574180 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59884: BlockingRecv error: recv got EOF from 127.0.0.1:59884 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 06:40:11.578578 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.577668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.30.71.61:32861)
0810 06:40:11.577713 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.577715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.577990 (+ 275us) server_negotiation.cc:407] Connection header received
0810 06:40:11.578103 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.578104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.578118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.578151 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.578521 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59898: BlockingRecv error: recv got EOF from 127.0.0.1:59898 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 06:40:11.585059 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.584056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.30.71.61:32861)
0810 06:40:11.584132 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.584134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.584421 (+ 287us) server_negotiation.cc:407] Connection header received
0810 06:40:11.584534 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.584535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.584548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.584582 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.584994 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59902: BlockingRecv error: recv got EOF from 127.0.0.1:59902 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 06:40:11.596032 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.594554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.30.71.61:32861)
0810 06:40:11.594607 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.594609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.594743 (+ 134us) server_negotiation.cc:407] Connection header received
0810 06:40:11.594824 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.594825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.594837 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.594883 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.595939 (+ 1056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59912: BlockingRecv error: recv got EOF from 127.0.0.1:59912 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 06:40:11.613303 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.612501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.30.71.61:32861)
0810 06:40:11.612550 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.612552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.612693 (+ 141us) server_negotiation.cc:407] Connection header received
0810 06:40:11.612783 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.612784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.612803 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.612873 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.613228 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59920: BlockingRecv error: recv got EOF from 127.0.0.1:59920 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 06:40:11.647513 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.646597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.30.71.61:32861)
0810 06:40:11.646664 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.646667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.646842 (+ 175us) server_negotiation.cc:407] Connection header received
0810 06:40:11.646941 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.646942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.646955 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.647030 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.647435 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59928: BlockingRecv error: recv got EOF from 127.0.0.1:59928 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 06:40:11.713785 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.713007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.30.71.61:32861)
0810 06:40:11.713087 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.713089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.713155 (+ 66us) server_negotiation.cc:407] Connection header received
0810 06:40:11.713230 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.713231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.713243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.713307 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.713681 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59940: BlockingRecv error: recv got EOF from 127.0.0.1:59940 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 06:40:11.843946 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:11.843178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59952 (local address 127.30.71.61:32861)
0810 06:40:11.843252 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:11.843254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:11.843306 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:40:11.843381 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:11.843382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:11.843394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:11.843465 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:11.843883 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59952: BlockingRecv error: recv got EOF from 127.0.0.1:59952 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 06:40:12.102138 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:12.101384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59964 (local address 127.30.71.61:32861)
0810 06:40:12.101468 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:12.101471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:12.101484 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:40:12.101531 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:12.101532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:12.101554 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:12.101637 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:12.102042 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59964: BlockingRecv error: recv got EOF from 127.0.0.1:59964 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:40:12.377007 22869 catalog_manager.cc:1259] Loaded cluster ID: a926ced8addc463db10799de14a4656b
I20260810 06:40:12.377099 22869 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: loading cluster ID for follower catalog manager: success
I20260810 06:40:12.378059 22869 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: acquiring CA information for follower catalog manager: success
I20260810 06:40:12.378468 22869 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 780dcb8b1aa34a69801255cead4f3d45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:40:12.420233 22886 catalog_manager.cc:1259] Loaded cluster ID: a926ced8addc463db10799de14a4656b
I20260810 06:40:12.420320 22886 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: loading cluster ID for follower catalog manager: success
I20260810 06:40:12.421089 22886 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: acquiring CA information for follower catalog manager: success
I20260810 06:40:12.421343 22886 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71d234694d91498e9e46099eaee5f0df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:40:12.618508 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:12.615495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.30.71.61:32861)
0810 06:40:12.615630 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:12.615632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:12.615781 (+ 149us) server_negotiation.cc:407] Connection header received
0810 06:40:12.615819 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:12.615820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:12.615879 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:12.615977 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:12.616174 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.616677 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.617513 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.617741 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.617816 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:12.618070 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:12.618194 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:12.618277 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:12.618402 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 06:40:12.618426 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260810 06:40:12.623332 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:12.620595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.30.71.60:39299)
0810 06:40:12.620787 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:12.620789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:12.620798 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:40:12.620840 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:12.620840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:12.620881 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:12.620960 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:12.621051 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.621590 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.622320 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.622539 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.622605 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:12.622841 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:12.622954 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:12.623154 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:12.623195 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 06:40:12.623221 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":117,"threads_started":1}
I20260810 06:40:12.628505 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:12.625326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.30.71.61:32861)
0810 06:40:12.625385 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:12.625387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:12.625582 (+ 195us) server_negotiation.cc:407] Connection header received
0810 06:40:12.625695 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:12.625696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:12.625709 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:12.625774 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:12.625987 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.626764 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.627647 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.627939 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.627980 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:12.628193 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:12.628228 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:12.628281 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:12.628383 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 06:40:12.628411 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:40:12.628635 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:12.625720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.30.71.60:39299)
0810 06:40:12.625762 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:12.625764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:12.625922 (+ 158us) server_negotiation.cc:407] Connection header received
0810 06:40:12.625956 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:12.625957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:12.625968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:12.626031 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:12.626148 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.626846 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.627826 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.628107 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.628165 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:12.628347 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:12.628374 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:12.628420 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:12.628516 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 06:40:12.628536 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 06:40:12.628961 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:12.625316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.30.71.62:36279)
0810 06:40:12.625451 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:12.625454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:12.625523 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:40:12.625614 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:12.625615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:12.625695 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:12.625804 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:12.626001 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.626555 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.627599 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:12.627921 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:12.627999 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:12.628199 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:12.628356 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:12.628415 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:12.628486 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 06:40:12.628512 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260810 06:40:12.630710 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22615
I20260810 06:40:12.639398 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22697
I20260810 06:40:12.647361 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 22778
2026-08-10T06:40:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3015 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:40:12 dist-test-slave-w206 krb5kdc[23011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:40:12 dist-test-slave-w206 krb5kdc[23011](info): set up 2 sockets
Aug 10 06:40:12 dist-test-slave-w206 krb5kdc[23011](info): commencing operation
krb5kdc: starting...
W20260810 06:40:13.759595 31004 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/test-user.keytab.
Aug 10 06:40:13 dist-test-slave-w206 krb5kdc[23011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:40:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:40:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.71.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
Entry for principal kudu/127.30.71.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab.
I20260810 06:40:13.893816 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:46481
--webserver_interface=127.30.71.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:46481,127.30.71.61:32791,127.30.71.60:38653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:40:13.979956 23024 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:40:13.980197 23024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:13.980216 23024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:13.980238 23024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:13.981459 23024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:13.981506 23024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:13.981520 23024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:13.981534 23024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:13.981546 23024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:13.983140 23024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:46481,127.30.71.61:32791,127.30.71.60:38653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.62.keytab
--principal=kudu/127.30.71.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.62:46481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.71.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:13.983423 23024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:13.983673 23024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:13.988919 23032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:13.988965 23030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:13.988915 23029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:13.989269 23024 server_base.cc:1034] running on GCE node
I20260810 06:40:13.989529 23024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:13.989784 23024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:13.990953 23024 hybrid_clock.cc:648] HybridClock initialized: now 1786344013990939 us; error 31 us; skew 500 ppm
Aug 10 06:40:13 dist-test-slave-w206 krb5kdc[23011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:13.993966 23024 init.cc:377] Logged in from keytab as kudu/127.30.71.62@KRBTEST.COM (short username kudu)
I20260810 06:40:13.994509 23024 webserver.cc:460] Webserver started at http://127.30.71.62:46151/ using document root <none> and password file <none>
I20260810 06:40:13.994791 23024 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:13.994844 23024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:13.995044 23024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:13.996302 23024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data/instance:
uuid: "d32f0e646a974432908c77589842375a"
format_stamp: "Formatted at 2026-08-10 06:40:13 on dist-test-slave-w206"
server_key: "201bdfe73ba74141cca1dc476aa1d542"
server_key_iv: "82d0dc3879189b9297313348e80b7ad7"
server_key_version: "encryptionkey@0"
I20260810 06:40:13.996727 23024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/wal/instance:
uuid: "d32f0e646a974432908c77589842375a"
format_stamp: "Formatted at 2026-08-10 06:40:13 on dist-test-slave-w206"
server_key: "201bdfe73ba74141cca1dc476aa1d542"
server_key_iv: "82d0dc3879189b9297313348e80b7ad7"
server_key_version: "encryptionkey@0"
I20260810 06:40:13.998257 23024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 06:40:13.999205 23039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:13.999418 23024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:13.999526 23024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/wal
uuid: "d32f0e646a974432908c77589842375a"
format_stamp: "Formatted at 2026-08-10 06:40:13 on dist-test-slave-w206"
server_key: "201bdfe73ba74141cca1dc476aa1d542"
server_key_iv: "82d0dc3879189b9297313348e80b7ad7"
server_key_version: "encryptionkey@0"
I20260810 06:40:13.999614 23024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:14.018002 23024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:14.020701 23024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:14.020918 23024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:14.025221 23024 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.62:46481
I20260810 06:40:14.025259 23094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.62:46481 every 8 connection(s)
I20260810 06:40:14.025635 23024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/data/info.pb
I20260810 06:40:14.026646 23095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:14.028342 23095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:14.029911 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 23024
I20260810 06:40:14.030022 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-0/wal/instance
I20260810 06:40:14.030232 31004 external_mini_cluster.cc:1428] Setting key 0a31f5cd118d6b6be68bf66d408bff68
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.71.61@KRBTEST.COM; defaulting to no policy
I20260810 06:40:14.040519 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.029138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49088 (local address 127.30.71.62:46481)
0810 06:40:14.029322 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.029332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.029439 (+ 107us) server_negotiation.cc:407] Connection header received
0810 06:40:14.029980 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.029983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.030011 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.030124 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.030757 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.032083 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.033430 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.033960 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.034084 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.036667 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.036678 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.036678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.036687 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.038663 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.039113 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.039116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.039117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.039245 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.039484 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.039486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.039486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.039820 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.039968 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.040189 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.040362 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260810 06:40:14.040817 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.029095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:46481 (local address 127.0.0.1:49088)
0810 06:40:14.029293 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.029343 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.029808 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.030136 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.030139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.030484 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.030653 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.030658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.032194 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.032195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.033304 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.033307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.034078 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.034079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.034136 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.034734 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.034743 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.036564 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.038755 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.038762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.038764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.039017 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.039329 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.039329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.039329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.039416 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.039961 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.039973 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.040042 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.040134 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.040298 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
I20260810 06:40:14.041693 23095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:14.042138 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.041949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42098)
0810 06:40:14.042018 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.042103 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:40:14.042346 23040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
I20260810 06:40:14.042860 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.042772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42104)
0810 06:40:14.042813 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.042836 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.61@KRBTEST.COM" created.
W20260810 06:40:14.043572 23095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:32791: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab.
Entry for principal kudu/127.30.71.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab.
I20260810 06:40:14.058508 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:32791
--webserver_interface=127.30.71.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab
--principal=kudu/127.30.71.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:46481,127.30.71.61:32791,127.30.71.60:38653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:14.092845 23095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } attempt: 1
I20260810 06:40:14.093413 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.093262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42116)
0810 06:40:14.093331 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.093373 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:40:14.093875 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.093775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42132)
0810 06:40:14.093831 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.093857 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:40:14.093998 23095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:32791: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
W20260810 06:40:14.143383 23104 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:40:14.143707 23104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:14.143728 23104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:14.143752 23104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:14.144917 23104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:14.144960 23104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:14.144974 23104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:14.144987 23104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:14.144999 23104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:14.146548 23104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:46481,127.30.71.61:32791,127.30.71.60:38653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.61.keytab
--principal=kudu/127.30.71.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.61:32791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.71.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:14.146775 23104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:14.146966 23104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:40:14.152204 23104 server_base.cc:1034] running on GCE node
W20260810 06:40:14.152156 23110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:14.152140 23109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:14.152119 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:14.152592 23104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:14.152920 23104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:14.154070 23104 hybrid_clock.cc:648] HybridClock initialized: now 1786344014154049 us; error 36 us; skew 500 ppm
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:14.156607 23104 init.cc:377] Logged in from keytab as kudu/127.30.71.61@KRBTEST.COM (short username kudu)
I20260810 06:40:14.157078 23104 webserver.cc:460] Webserver started at http://127.30.71.61:37129/ using document root <none> and password file <none>
I20260810 06:40:14.157338 23104 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:14.157387 23104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:14.157552 23104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:14.158669 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data/instance:
uuid: "b069d9090af14b2cb22d782e8523448e"
format_stamp: "Formatted at 2026-08-10 06:40:14 on dist-test-slave-w206"
server_key: "fbae64b47916035e3fcd4ecbfb2d6d1f"
server_key_iv: "d49d3986a626ab579be349570b4c3e0f"
server_key_version: "encryptionkey@0"
I20260810 06:40:14.159117 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/wal/instance:
uuid: "b069d9090af14b2cb22d782e8523448e"
format_stamp: "Formatted at 2026-08-10 06:40:14 on dist-test-slave-w206"
server_key: "fbae64b47916035e3fcd4ecbfb2d6d1f"
server_key_iv: "d49d3986a626ab579be349570b4c3e0f"
server_key_version: "encryptionkey@0"
I20260810 06:40:14.160542 23104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:14.161399 23120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:14.161603 23104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:40:14.161712 23104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/wal
uuid: "b069d9090af14b2cb22d782e8523448e"
format_stamp: "Formatted at 2026-08-10 06:40:14 on dist-test-slave-w206"
server_key: "fbae64b47916035e3fcd4ecbfb2d6d1f"
server_key_iv: "d49d3986a626ab579be349570b4c3e0f"
server_key_version: "encryptionkey@0"
I20260810 06:40:14.161787 23104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:14.162177 23095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } attempt: 2
I20260810 06:40:14.162705 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.162553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42134)
0810 06:40:14.162622 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.162667 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:40:14.163173 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.163053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42146)
0810 06:40:14.163116 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.163149 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:40:14.163300 23095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.61:32791: Network error: Client connection negotiation failed: client connection to 127.30.71.61:32791: connect: Connection refused (error 111)
I20260810 06:40:14.188704 23104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:14.191512 23104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:14.191691 23104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:14.196715 23104 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.61:32791
I20260810 06:40:14.196776 23177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.61:32791 every 8 connection(s)
I20260810 06:40:14.197186 23104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/data/info.pb
I20260810 06:40:14.198385 23179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:14.200135 23179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:14.204622 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 23104
I20260810 06:40:14.204717 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-1/wal/instance
I20260810 06:40:14.204916 31004 external_mini_cluster.cc:1428] Setting key d1844e9e533c297415e764e1d1074735
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
I20260810 06:40:14.210024 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.200913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.30.71.62:46481)
0810 06:40:14.201006 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.201009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.201185 (+ 176us) server_negotiation.cc:407] Connection header received
0810 06:40:14.201667 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.201668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.201681 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.201734 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.202498 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.203043 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.204133 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.204358 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.204437 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.206730 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.206732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.206733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.206741 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.208499 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.208939 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.208941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.208941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.209015 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.209275 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.209277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.209277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.209456 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.209543 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.209824 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.209922 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:40:14.210244 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.200846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:46481 (local address 127.0.0.1:49094)
0810 06:40:14.201057 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.201098 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.201580 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.201790 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.201793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.202137 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.202369 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.202373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.203150 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.203152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.204057 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.204060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.204462 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.204463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.204570 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.205190 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.205198 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.206624 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.208592 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.208600 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.208602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.208859 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.209087 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.209088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.209088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.209192 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.209534 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.209541 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.209686 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.209802 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.209992 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":112,"threads_started":1}
I20260810 06:40:14.210891 23179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.71.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.71.60@KRBTEST.COM" created.
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:40:14.220884 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.211228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42154)
0810 06:40:14.211283 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.211295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.211377 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.211623 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.211624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.211776 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.211831 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.211832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.212676 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.212678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.213170 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.213172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.213745 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.213746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.213810 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.214286 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.214291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.217347 (+ 3056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.219720 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.219721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.219722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.219917 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.220144 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.220145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.220145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.220199 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.220581 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.220582 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.220629 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.220718 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.220807 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:40:14.221065 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.211280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.30.71.61:32791)
0810 06:40:14.211479 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.211483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.211511 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:40:14.211528 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.211530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.211540 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.211623 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.211910 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.212585 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.213267 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.213658 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.213737 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.217454 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.217465 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.217465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.217473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.219607 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.220000 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.220005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.220005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.220074 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.220258 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.220259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.220259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.220480 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.220570 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.220769 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.220904 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":171,"thread_start_us":121,"threads_started":1}
I20260810 06:40:14.221925 23179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:14.222388 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.222194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45740)
0810 06:40:14.222299 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.222358 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:40:14.222529 23121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
I20260810 06:40:14.223011 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.222899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45746)
0810 06:40:14.222945 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.222970 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 06:40:14.223126 23179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:38653: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.71.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab.
Entry for principal kudu/127.30.71.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab.
I20260810 06:40:14.235716 31004 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskI3NzPm/build/release/bin/kudu
/tmp/dist-test-taskI3NzPm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:38653
--webserver_interface=127.30.71.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab
--principal=kudu/127.30.71.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.71.20:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.71.62:46481,127.30.71.61:32791,127.30.71.60:38653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:40:14.254513 23095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } attempt: 3
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:40:14.262020 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.255075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.30.71.61:32791)
0810 06:40:14.255153 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.255156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.255165 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:40:14.255195 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.255196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.255214 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.255281 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.255640 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.256166 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.256924 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.257156 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.257236 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.259253 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.259255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.259255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.259264 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.260791 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.261188 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.261190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.261190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.261249 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.261468 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.261469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.261469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.261590 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.261668 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.261832 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.261918 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:40:14.262020 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.254965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42164)
0810 06:40:14.255052 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.255069 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.255130 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.255300 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.255301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.255485 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.255552 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.255554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.256275 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.256276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.256828 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.256830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.257246 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.257248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.257336 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.257829 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.257833 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.259153 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.260911 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.260912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.260913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.261099 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.261328 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.261329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.261329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.261398 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.261660 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.261661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.261712 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.261802 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.261915 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 06:40:14.262394 23095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:14.262756 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.262650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45750)
0810 06:40:14.262700 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.262730 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 06:40:14.263159 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.263087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45760)
0810 06:40:14.263120 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.263143 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 06:40:14.263262 23095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:38653: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
I20260810 06:40:14.272380 23179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } attempt: 1
I20260810 06:40:14.272850 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.272711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45768)
0810 06:40:14.272773 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.272813 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:40:14.273197 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.273100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45782)
0810 06:40:14.273147 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.273172 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 06:40:14.273279 23179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:38653: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
I20260810 06:40:14.294436 23095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } attempt: 1
I20260810 06:40:14.295082 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.294883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45784)
0810 06:40:14.294956 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.295051 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:40:14.295449 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.295348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45800)
0810 06:40:14.295395 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.295421 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 06:40:14.295545 23095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:38653: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
W20260810 06:40:14.322921 23186 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:40:14.323233 23186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:40:14.323252 23186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:40:14.323275 23186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:40:14.324501 23186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:40:14.324548 23186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:40:14.324561 23186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:40:14.324574 23186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:40:14.324613 23186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:40:14.326156 23186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.71.20:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.71.62:46481,127.30.71.61:32791,127.30.71.60:38653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/krb5kdc/kudu_127.30.71.60.keytab
--principal=kudu/127.30.71.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.71.60:38653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.71.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 05:47:10 UTC on ccycloud.quasar-gieubf.root.comops.site
build id 4577114
I20260810 06:40:14.326442 23186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:40:14.326709 23186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:40:14.331890 23194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:14.331885 23192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:40:14.331885 23196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:40:14.332216 23186 server_base.cc:1034] running on GCE node
I20260810 06:40:14.332468 23186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:40:14.332731 23186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:40:14.333873 23186 hybrid_clock.cc:648] HybridClock initialized: now 1786344014333862 us; error 31 us; skew 500 ppm
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:40:14.336621 23186 init.cc:377] Logged in from keytab as kudu/127.30.71.60@KRBTEST.COM (short username kudu)
I20260810 06:40:14.337039 23186 webserver.cc:460] Webserver started at http://127.30.71.60:36653/ using document root <none> and password file <none>
I20260810 06:40:14.337239 23186 fs_manager.cc:359] Metadata directory not provided
I20260810 06:40:14.337280 23186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:40:14.337383 23186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:40:14.338423 23186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data/instance:
uuid: "99eb809f3fa94fc89f80b031d3aae2c6"
format_stamp: "Formatted at 2026-08-10 06:40:14 on dist-test-slave-w206"
server_key: "e8f836292b4ba91e7837bf65623cdd3f"
server_key_iv: "b8aca29f5d4583d68b54112bf25b72ba"
server_key_version: "encryptionkey@0"
I20260810 06:40:14.338729 23186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/wal/instance:
uuid: "99eb809f3fa94fc89f80b031d3aae2c6"
format_stamp: "Formatted at 2026-08-10 06:40:14 on dist-test-slave-w206"
server_key: "e8f836292b4ba91e7837bf65623cdd3f"
server_key_iv: "b8aca29f5d4583d68b54112bf25b72ba"
server_key_version: "encryptionkey@0"
I20260810 06:40:14.340116 23186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 06:40:14.340868 23204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:14.341037 23186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 06:40:14.341146 23186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data,/tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/wal
uuid: "99eb809f3fa94fc89f80b031d3aae2c6"
format_stamp: "Formatted at 2026-08-10 06:40:14 on dist-test-slave-w206"
server_key: "e8f836292b4ba91e7837bf65623cdd3f"
server_key_iv: "b8aca29f5d4583d68b54112bf25b72ba"
server_key_version: "encryptionkey@0"
I20260810 06:40:14.341224 23186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:40:14.341419 23179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } attempt: 2
I20260810 06:40:14.341838 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.341716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45804)
0810 06:40:14.341760 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.341807 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 06:40:14.342175 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.342075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45814)
0810 06:40:14.342126 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.342160 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:40:14.342306 23179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.71.60:38653: Network error: Client connection negotiation failed: client connection to 127.30.71.60:38653: connect: Connection refused (error 111)
I20260810 06:40:14.361599 23186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:40:14.364132 23186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:40:14.364262 23186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:40:14.368561 23186 rpc_server.cc:307] RPC server started. Bound to: 127.30.71.60:38653
I20260810 06:40:14.368677 23258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.71.60:38653 every 8 connection(s)
I20260810 06:40:14.369014 23186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/data/info.pb
I20260810 06:40:14.370132 23259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:40:14.370684 23095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } attempt: 2
I20260810 06:40:14.372218 23259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:14.372648 31004 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskI3NzPm/build/release/bin/kudu as pid 23186
I20260810 06:40:14.372777 31004 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskI3NzPm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786343734291041-31004-0/minicluster-data/master-2/wal/instance
I20260810 06:40:14.373008 31004 external_mini_cluster.cc:1428] Setting key c2d21c0301618334521d954f4816f715
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.62@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
I20260810 06:40:14.381975 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.371035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45818)
0810 06:40:14.371093 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.371108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.371159 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.372217 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.372218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.372398 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.372445 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.372470 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.374430 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.374431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.375202 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.375204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.375983 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.375984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.376042 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.376546 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.376552 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.378026 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.380439 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.380440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.380440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.380655 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.381150 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.381151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.381152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.381225 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.381664 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.381665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.381713 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.381763 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.381864 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:40:14.382164 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.372918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.62:46481 (local address 127.0.0.1:49102)
0810 06:40:14.373113 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.373148 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.373221 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.373606 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.373609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.373961 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.374053 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.374058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.374965 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.374966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.375667 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.375669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.376226 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.376228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.376296 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.376721 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.376729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.378510 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.380273 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.380280 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.380282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.380530 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.380736 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.380737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.380737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.380891 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.381315 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.381378 (+ 63us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.381578 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.381700 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.381892 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260810 06:40:14.382473 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.371209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.30.71.60:38653)
0810 06:40:14.371414 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.371424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.371460 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:40:14.371986 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.371994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.372059 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.372218 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.372598 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.374058 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.375287 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.375899 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.376020 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.378117 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.378125 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.378126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.378133 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.380313 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.380998 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.381002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.381003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.381055 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.381317 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.381318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.381318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.381570 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.382204 (+ 634us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.382255 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.382360 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260810 06:40:14.383320 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.373329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45820 (local address 127.30.71.60:38653)
0810 06:40:14.373547 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.373549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.373557 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:40:14.373635 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.373635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.373644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.373708 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.374399 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.374874 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.375866 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.376181 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.376242 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.379140 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.379141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.379141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.379148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.381851 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.382248 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.382249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.382249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.382325 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.382529 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.382530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.382530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.382649 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.382761 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.383031 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.383130 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
I20260810 06:40:14.383647 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.372948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.30.71.62:46481)
0810 06:40:14.372993 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.372995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.373209 (+ 214us) server_negotiation.cc:407] Connection header received
0810 06:40:14.373468 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.373469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.373480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.373544 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.374163 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.374878 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.375758 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.376049 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.376142 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.378604 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.378606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.378606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.378614 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.380187 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.380601 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.380602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.380603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.380655 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.380975 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.380977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.380977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.381203 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.381283 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.382444 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.383529 (+ 1085us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 06:40:14.384054 23259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:40:14.384281 23095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a: Bootstrap starting.
I20260810 06:40:14.385350 23095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:14.385650 23095 log.cc:824] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:14.386579 23095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a: No bootstrap required, opened a new log
I20260810 06:40:14.388912 23095 raft_consensus.cc:348] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
I20260810 06:40:14.389058 23095 raft_consensus.cc:729] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d32f0e646a974432908c77589842375a, State: Initialized, Role: FOLLOWER
I20260810 06:40:14.389145 23095 consensus_queue.cc:260] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:40:14.389518 23095 sys_catalog.cc:564] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:14.390616 23266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.390723 23266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: This master's current role is: FOLLOWER
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.62@KRBTEST.COM
W20260810 06:40:14.392374 23279 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:14.392431 23279 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:14.393220 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.384331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.61:32791 (local address 127.0.0.1:42168)
0810 06:40:14.384386 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.384398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.384444 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.384589 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.384589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.384705 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.384757 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.384759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.385640 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.385640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.386294 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.386297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.386838 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.386839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.386911 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.388173 (+ 1262us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.388178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.389687 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.391728 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.391729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.391730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.392251 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.392468 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.392469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.392469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.392543 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.392887 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.392888 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.392941 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.393013 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.393117 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:40:14.393756 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.384418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.30.71.61:32791)
0810 06:40:14.384464 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.384466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.384476 (+ 10us) server_negotiation.cc:407] Connection header received
0810 06:40:14.384502 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.384503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.384514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.384572 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.384845 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.385557 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.386454 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.386754 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.386839 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.389822 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.389823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.389824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.389832 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.391643 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.392334 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.392335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.392336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.392404 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.392616 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.392617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.392617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.392765 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.392854 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.393576 (+ 722us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.393660 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 06:40:14.394753 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.386566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.30.71.62:46481)
0810 06:40:14.386602 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.386604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.386612 (+ 8us) server_negotiation.cc:407] Connection header received
0810 06:40:14.386662 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.386663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.386673 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.386737 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.387369 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.388033 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.388746 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.388950 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.389036 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.391696 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.391697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.391698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.391705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.393541 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.393958 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.393959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.393959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.394013 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.394232 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.394233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.394234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.394354 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.394503 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.394582 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.394661 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:40:14.392436 23024 master_runner.cc:186] Error getting master registration for 127.30.71.61:32791: OK, instance_id { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" instance_seqno: 1786344014195257 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:40:14.395764 23024 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:40:14.396319 23259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.60@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.71.61@KRBTEST.COM
I20260810 06:40:14.404299 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.396637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45822)
0810 06:40:14.396706 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.396716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.396758 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.396911 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.396912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.397038 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.397061 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.397063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.397716 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.397717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.398477 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.398480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.398966 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.398967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.399037 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.399320 (+ 283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.399325 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.401296 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.403186 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.403187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.403188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.403387 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.403595 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.403596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.403597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.403652 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.403955 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.403956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.403997 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.404060 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.404177 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 06:40:14.404613 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.396671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45822 (local address 127.30.71.60:38653)
0810 06:40:14.396710 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.396712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.396745 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:40:14.396813 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.396814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.396824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.396876 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.397150 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.397627 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.398568 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.398882 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.398969 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.401392 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.401393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.401394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.401400 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.403078 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.403467 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.403468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.403468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.403522 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.403726 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.403727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.403727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.403884 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.403964 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.404501 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.404561 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 06:40:14.404888 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.396048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.30.71.61:32791)
0810 06:40:14.396122 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.396124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.396136 (+ 12us) server_negotiation.cc:407] Connection header received
0810 06:40:14.396157 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.396158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.396168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.396234 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.396873 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.397589 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.398282 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.398502 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.398580 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.401497 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.401498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.401499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.401507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.403405 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.403814 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.403815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.403816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.403872 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.404075 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.404076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.404076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.404218 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.404384 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.404711 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.404806 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:40:14.406315 23259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: Bootstrap starting.
I20260810 06:40:14.406736 31004 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:40:14.407163 23259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:14.407389 23259 log.cc:824] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:14.407713 23067 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d32f0e646a974432908c77589842375a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49108
I20260810 06:40:14.407891 23067 raft_consensus.cc:482] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:40:14.407938 23067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:14.408118 23259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: No bootstrap required, opened a new log
I20260810 06:40:14.408871 23067 raft_consensus.cc:504] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
I20260810 06:40:14.409129 23067 leader_election.cc:288] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [CANDIDATE]: Term 1 election: Requested vote from peers b069d9090af14b2cb22d782e8523448e (127.30.71.61:32791), 99eb809f3fa94fc89f80b031d3aae2c6 (127.30.71.60:38653)
I20260810 06:40:14.409806 23259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
I20260810 06:40:14.409972 23259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb809f3fa94fc89f80b031d3aae2c6, State: Initialized, Role: FOLLOWER
I20260810 06:40:14.410059 23259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
I20260810 06:40:14.410426 23259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:14.410496 23232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d32f0e646a974432908c77589842375a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "99eb809f3fa94fc89f80b031d3aae2c6"
I20260810 06:40:14.410640 23232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:14.411765 23232 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d32f0e646a974432908c77589842375a in term 1.
I20260810 06:40:14.412092 23040 leader_election.cc:302] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99eb809f3fa94fc89f80b031d3aae2c6, d32f0e646a974432908c77589842375a; no voters:
I20260810 06:40:14.412274 23266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:40:14.412469 23266 raft_consensus.cc:686] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [term 1 LEADER]: Becoming Leader. State: Replica: d32f0e646a974432908c77589842375a, State: Running, Role: LEADER
I20260810 06:40:14.412590 23266 consensus_queue.cc:237] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
I20260810 06:40:14.413055 23266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d32f0e646a974432908c77589842375a. Latest consensus state: current_term: 1 leader_uuid: "d32f0e646a974432908c77589842375a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.413136 23266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:14.413420 23308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:40:14.412573 23287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.413605 23308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:40:14.413609 23287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:40:14.412892 23304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:40:14.413825 23304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:40:14.415200 23186 master_runner.cc:186] Error getting master registration for 127.30.71.61:32791: OK, instance_id { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" instance_seqno: 1786344014195257 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:40:14.417758 23186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:40:14.425288 23232 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [term 1 FOLLOWER]: Refusing update from remote peer d32f0e646a974432908c77589842375a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:14.425752 23266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:14.426899 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.411015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.30.71.60:38653)
0810 06:40:14.411921 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.411924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.417829 (+ 5905us) server_negotiation.cc:407] Connection header received
0810 06:40:14.417854 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.417855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.417867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.417927 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.418706 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.419429 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.420746 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.421056 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.421103 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.423141 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.423142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.423143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.423151 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.425138 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.425674 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.425675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.425676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.425737 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.425950 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.425951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.425951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.426083 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.426186 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.426411 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.426501 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878}
I20260810 06:40:14.427273 23287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d32f0e646a974432908c77589842375a. Latest consensus state: current_term: 1 leader_uuid: "d32f0e646a974432908c77589842375a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.427381 23287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:14.429456 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.410325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.30.71.62:46481)
0810 06:40:14.410371 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.410373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.415453 (+ 5080us) server_negotiation.cc:407] Connection header received
0810 06:40:14.415542 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.415543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.415553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.415582 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.416272 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.417028 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.423081 (+ 6053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.423393 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.423437 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.426384 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.426385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.426385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.426393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.428135 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.428569 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.428570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.428570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.428625 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.428823 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.428824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.428824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.428957 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.429034 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.429276 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.429362 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:40:14.429519 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.410538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.30.71.61:32791)
0810 06:40:14.410595 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.410597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.415144 (+ 4547us) server_negotiation.cc:407] Connection header received
0810 06:40:14.415277 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.415278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.415290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.415346 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.416600 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.417327 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.419123 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.419435 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.419478 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.422009 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.422011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.422011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.422019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.424306 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.428438 (+ 4132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.428440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.428440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.428522 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.428854 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.428855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.428856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.428981 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.429057 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.429328 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.429426 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 06:40:14.430711 23266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d32f0e646a974432908c77589842375a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.430817 23266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: This master's current role is: LEADER
I20260810 06:40:14.430912 23308 catalog_manager.cc:1347] Generated new cluster ID: 4f0b7c6fea70412985d99a8cd236e105
I20260810 06:40:14.430943 23308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:40:14.433442 23179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } attempt: 3
I20260810 06:40:14.435866 23287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d32f0e646a974432908c77589842375a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.435967 23287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6 [sys.catalog]: This master's current role is: FOLLOWER
Aug 10 06:40:14 dist-test-slave-w206 krb5kdc[23011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344014, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.71.61@KRBTEST.COM for kudu/127.30.71.60@KRBTEST.COM
I20260810 06:40:14.442718 23310 mvcc.cc:205] Tried to move back new op lower bound from 7316865083081609216 to 7316865083034816512. Current Snapshot: MvccSnapshot[applied={T|T < 7316865083081609216 or (T in {7316865083081609216})}]
I20260810 06:40:14.443154 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.435042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.30.71.60:38653)
0810 06:40:14.435097 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.435099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.435108 (+ 9us) server_negotiation.cc:407] Connection header received
0810 06:40:14.435135 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.435137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.435148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.435210 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.435537 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.436222 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.437193 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.437458 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.437531 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.439850 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:40:14.439852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:40:14.439852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:40:14.439859 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:40:14.441676 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.442114 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.442115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.442116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.442189 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:40:14.442372 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:40:14.442373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:40:14.442373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:40:14.442534 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:40:14.442599 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:14.442935 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 06:40:14.443045 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:40:14.443866 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:14.433762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.71.60:38653 (local address 127.0.0.1:45842)
0810 06:40:14.433818 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 06:40:14.433829 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 06:40:14.433870 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:40:14.435216 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:40:14.435217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:40:14.435420 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:40:14.435471 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.435473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.436313 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.436314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.437119 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:40:14.437121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:40:14.437544 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:40:14.437545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:40:14.437619 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:14.438133 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:40:14.438138 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:40:14.439742 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:40:14.441779 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.441780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.441781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.442030 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.442243 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:40:14.442244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:40:14.442245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:40:14.442322 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:40:14.442627 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:40:14.442628 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:40:14.442689 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 06:40:14.442759 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:40:14.443771 (+ 1012us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:40:14.443411 23311 mvcc.cc:205] Tried to move back new op lower bound from 7316865083081609216 to 7316865083034816512. Current Snapshot: MvccSnapshot[applied={T|T < 7316865083081609216}]
I20260810 06:40:14.445861 23179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e: Bootstrap starting.
I20260810 06:40:14.446655 23179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e: Neither blocks nor log segments found. Creating new log.
I20260810 06:40:14.447065 23179 log.cc:824] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:40:14.448171 23179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e: No bootstrap required, opened a new log
I20260810 06:40:14.450403 23179 raft_consensus.cc:348] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
I20260810 06:40:14.450568 23179 raft_consensus.cc:729] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b069d9090af14b2cb22d782e8523448e, State: Initialized, Role: FOLLOWER
I20260810 06:40:14.450682 23179 consensus_queue.cc:260] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } }
I20260810 06:40:14.451140 23179 sys_catalog.cc:564] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:40:14.451244 23147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:40:14.452328 23147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [term 1 FOLLOWER]: Refusing update from remote peer d32f0e646a974432908c77589842375a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:40:14.452548 23148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d32f0e646a974432908c77589842375a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b069d9090af14b2cb22d782e8523448e"
I20260810 06:40:14.453462 23148 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d32f0e646a974432908c77589842375a in term 1.
I20260810 06:40:14.453898 23314 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:40:14.455037 23308 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:40:14.455945 23308 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:40:14.456552 23336 sys_catalog.cc:455] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d32f0e646a974432908c77589842375a. Latest consensus state: current_term: 1 leader_uuid: "d32f0e646a974432908c77589842375a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.456686 23336 sys_catalog.cc:458] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:14.456753 23336 sys_catalog.cc:455] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d32f0e646a974432908c77589842375a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.456790 23336 sys_catalog.cc:458] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:14.451115 23318 sys_catalog.cc:455] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.457410 23318 sys_catalog.cc:458] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:40:14.459777 23330 catalog_manager.cc:1259] Loaded cluster ID: 4f0b7c6fea70412985d99a8cd236e105
I20260810 06:40:14.459921 23330 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e: loading cluster ID for follower catalog manager: success
I20260810 06:40:14.461261 23330 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e: acquiring CA information for follower catalog manager: success
W20260810 06:40:14.461480 23330 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260810 06:40:14.462399 23308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a: Generated new TSK 0
I20260810 06:40:14.462598 23308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:40:14.463967 23305 sys_catalog.cc:455] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d32f0e646a974432908c77589842375a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d32f0e646a974432908c77589842375a" member_type: VOTER last_known_addr { host: "127.30.71.62" port: 46481 } } peers { permanent_uuid: "b069d9090af14b2cb22d782e8523448e" member_type: VOTER last_known_addr { host: "127.30.71.61" port: 32791 } } peers { permanent_uuid: "99eb809f3fa94fc89f80b031d3aae2c6" member_type: VOTER last_known_addr { host: "127.30.71.60" port: 38653 } } }
I20260810 06:40:14.464094 23305 sys_catalog.cc:458] T 00000000000000000000000000000000 P d32f0e646a974432908c77589842375a [sys.catalog]: This master's current role is: LEADER
W20260810 06:40:14.487622 23098 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.485661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49132 (local address 127.30.71.62:46481)
0810 06:40:14.485712 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.485715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.486766 (+ 1051us) server_negotiation.cc:407] Connection header received
0810 06:40:14.486890 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.486891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.486905 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.486940 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.487541 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49132: BlockingRecv error: recv got EOF from 127.0.0.1:49132 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 06:40:14.488489 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.486127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.30.71.61:32791)
0810 06:40:14.486172 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.486174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.487903 (+ 1729us) server_negotiation.cc:407] Connection header received
0810 06:40:14.488012 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.488013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.488027 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.488059 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.488425 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42198: BlockingRecv error: recv got EOF from 127.0.0.1:42198 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 06:40:14.489286 23261 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.486497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.30.71.60:38653)
0810 06:40:14.486579 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.486581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.488734 (+ 2153us) server_negotiation.cc:407] Connection header received
0810 06:40:14.488840 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.488842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.488853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.488885 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.489212 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45852: BlockingRecv error: recv got EOF from 127.0.0.1:45852 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260810 06:40:14.492072 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.491132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.30.71.61:32791)
0810 06:40:14.491172 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.491174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.491486 (+ 312us) server_negotiation.cc:407] Connection header received
0810 06:40:14.491604 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.491605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.491634 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.491674 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.492019 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42208: BlockingRecv error: recv got EOF from 127.0.0.1:42208 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 06:40:14.495512 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.494594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.30.71.61:32791)
0810 06:40:14.494634 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.494636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.494883 (+ 247us) server_negotiation.cc:407] Connection header received
0810 06:40:14.495024 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.495025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.495038 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.495073 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.495456 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42220: BlockingRecv error: recv got EOF from 127.0.0.1:42220 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 06:40:14.500054 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.498909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42230 (local address 127.30.71.61:32791)
0810 06:40:14.498955 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.498957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.499365 (+ 408us) server_negotiation.cc:407] Connection header received
0810 06:40:14.499487 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.499488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.499503 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.499536 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.499993 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42230: BlockingRecv error: recv got EOF from 127.0.0.1:42230 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 06:40:14.506578 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.505599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.30.71.61:32791)
0810 06:40:14.505649 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.505651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.505908 (+ 257us) server_negotiation.cc:407] Connection header received
0810 06:40:14.506023 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.506024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.506037 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.506072 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.506516 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42232: BlockingRecv error: recv got EOF from 127.0.0.1:42232 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:40:14.517632 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.516482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.30.71.61:32791)
0810 06:40:14.516535 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.516538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.516846 (+ 308us) server_negotiation.cc:407] Connection header received
0810 06:40:14.516976 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.516977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.516993 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.517031 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.517559 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42244: BlockingRecv error: recv got EOF from 127.0.0.1:42244 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 06:40:14.536557 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.535434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42260 (local address 127.30.71.61:32791)
0810 06:40:14.535487 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.535489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.535804 (+ 315us) server_negotiation.cc:407] Connection header received
0810 06:40:14.535947 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.535949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.535963 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.535999 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.536488 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42260: BlockingRecv error: recv got EOF from 127.0.0.1:42260 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 06:40:14.571416 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.570360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.30.71.61:32791)
0810 06:40:14.570408 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.570410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.570701 (+ 291us) server_negotiation.cc:407] Connection header received
0810 06:40:14.570818 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.570819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.570832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.570866 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.571347 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42270: BlockingRecv error: recv got EOF from 127.0.0.1:42270 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 06:40:14.637985 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.637046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.30.71.61:32791)
0810 06:40:14.637105 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.637107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.637284 (+ 177us) server_negotiation.cc:407] Connection header received
0810 06:40:14.637332 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.637333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.637345 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.637435 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.637917 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42284: BlockingRecv error: recv got EOF from 127.0.0.1:42284 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260810 06:40:14.768088 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:14.767386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.30.71.61:32791)
0810 06:40:14.767482 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:14.767484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:14.767508 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:40:14.767567 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:14.767568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:14.767579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:14.767651 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:14.768037 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42290: BlockingRecv error: recv got EOF from 127.0.0.1:42290 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 06:40:15.026383 23183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:40:15.025676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.30.71.61:32791)
0810 06:40:15.025775 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:15.025777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:15.025791 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:40:15.025851 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:15.025852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:15.025867 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:15.025940 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:40:15.026312 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42306: BlockingRecv error: recv got EOF from 127.0.0.1:42306 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:40:15.414427 23304 catalog_manager.cc:1259] Loaded cluster ID: 4f0b7c6fea70412985d99a8cd236e105
I20260810 06:40:15.414515 23304 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: loading cluster ID for follower catalog manager: success
I20260810 06:40:15.415385 23304 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: acquiring CA information for follower catalog manager: success
I20260810 06:40:15.415663 23304 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 99eb809f3fa94fc89f80b031d3aae2c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:40:15.462240 23330 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b069d9090af14b2cb22d782e8523448e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:40:15.542743 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:15.539798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42308 (local address 127.30.71.61:32791)
0810 06:40:15.539953 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:15.539955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:15.539970 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:40:15.540075 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:15.540076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:15.540146 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:15.540237 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:15.540432 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.540952 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.541816 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.542059 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.542139 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:15.542325 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:15.542438 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:15.542522 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:15.542573 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 06:40:15.542618 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260810 06:40:15.547394 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:15.544793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.30.71.60:38653)
0810 06:40:15.544897 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:15.544899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:15.544956 (+ 57us) server_negotiation.cc:407] Connection header received
0810 06:40:15.545046 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:15.545046 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:15.545102 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:15.545192 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:15.545289 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.545786 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.546539 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.546756 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.546848 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:15.547020 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:15.547149 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:15.547242 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:15.547267 (+ 25us) server_negotiation.cc:299] Negotiation successful
0810 06:40:15.547292 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
I20260810 06:40:15.552520 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:15.549369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45872 (local address 127.30.71.60:38653)
0810 06:40:15.549407 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:15.549409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:15.549642 (+ 233us) server_negotiation.cc:407] Connection header received
0810 06:40:15.549699 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:15.549700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:15.549712 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:15.549760 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:15.549889 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.550617 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.551703 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.551950 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.552013 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:15.552191 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:15.552238 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:15.552268 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:15.552411 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 06:40:15.552434 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 06:40:15.552559 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:15.549098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.30.71.61:32791)
0810 06:40:15.549145 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:15.549147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:15.549395 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:40:15.549463 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:15.549464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:15.549475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:15.549550 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:15.549707 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.550508 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.551560 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.551845 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.551913 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:15.552056 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:15.552093 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:15.552145 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:15.552441 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 06:40:15.552471 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 06:40:15.552736 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:40:15.549164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.30.71.62:46481)
0810 06:40:15.549285 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 06:40:15.549287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 06:40:15.549325 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:40:15.549372 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:40:15.549373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:40:15.549430 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:40:15.549511 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:40:15.549657 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.550352 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.551600 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:40:15.551933 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:40:15.551988 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:40:15.552270 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:40:15.552393 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:40:15.552452 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 06:40:15.552621 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 06:40:15.552649 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260810 06:40:15.561179 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 23024
I20260810 06:40:15.568794 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 23104
I20260810 06:40:15.576133 31004 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskI3NzPm/build/release/bin/kudu with pid 23186
2026-08-10T06:40:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2929 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5945 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (281278 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST